how do you realize the network traffic controll stuff?

[markg85]

Hey,

i`m just wondering.. how do you grab all the network traffic and and grab out the parts you need to display how much traffic a certain (sub)somain has been used?

or.. just how do you do this stuff?
i`m just curious

Thanx in favor.

[ephigenie]

Traffic calculation is done through apache logs for http per domain,
ftp traffic is logged in mysql (proftpd) per user (then added)

and so on.

[markg85]

than you have a damn hard log analyzer..

this is a part of my apache access log file:
[quote]
127.0.0.1 - - [10/Dec/2006:16:20:06 +0100] "GET / HTTP/1.1" 200 19068 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1) Gecko/20061027 Firefox/2.0"
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/style.css HTTP/1.1" 200 2046 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1) Gec$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/homepage.gif HTTP/1.1" 200 2226 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/search.gif HTTP/1.1" 200 1990 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/oodomimagerollover.js HTTP/1.1" 200 3923 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; r$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/exed-banner.png HTTP/1.1" 200 22956 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US;$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/row_bg.gif HTTP/1.1" 200 218 "http://pafiledb353.mage.nl/skins/default/style.css" "Mozilla/5.0 (X11; U; Linu$
[/qoute]

how do you split all that stuff? on spaces? or better.. what`s the file where this is done? than i can have a look for myself

[ephigenie]

have a look at all that /var/log/apache2/<domain.tld>-traf.log files

[markg85]

http://httpd.apache.org/docs/2.2/mod/mod...onfig.html

handy i didn`t know that
now i`m gonna try and find out that proftpd stuff

you guys surely did a smart way with that traffic stuff.. i realy would have thought that all the network traffic was analyzed and splitted to the right domains.. but this is alot better

[markg85]

o.. just found this

http://bitbrook.de/software/mod_log_mysql/
and:
http://www.outoforder.cc/projects/apache/mod_log_sql/ (seems to be better)


with that you can store the log in mysql.. perhaps that`s a better way of doing it?
perhaps this can be used in the omega project or even in isp control panel.

[RatS]

if you are think it's a good idea, implement it!!

There are NO limits.

[ephigenie]

mod_log_mysql has its limits on busy hosts. But a good solution perhaps later on.
I've been using it about 2,5 years now on one of my servers.
But there're a lots of bugs in mod_log_sql and it is not ported cleanly to apache2.2.

(i speak of the mod_log_sql module) the developers promised one day to support the new ap2db - layer (apache2.2 can handle db - connections himself and keep a pool for modules like php and so on (only mod_php) )

The Traffic calculation can maybe one day be replaced with libapache-mod-acct-mysql if this one gets continued (at the moment only apache1 is support afaik)

in ISPCP we've to go even different ways, because we like to make the whole thing multi-server aware and aware of directing certain services to certain machines - such as a dedicated mail-server and so on..

[markg85]

o wow that`s cool
when i get some free time and when there is something of that ispcp released (builds/beta`s) i will probably start testing out some good stuff (your looking at a future team member here )

edit:://
this mod is also looking interesting
o and this bandwidth module is also looking very interesting now you don`t even have yo log the traffic (unless you want traffic graphics) becuase with this module you can set bandwidth limitations. even things like: files larger than 500kb are downloaded with speeds not faster than 100kb/s

[pierg75]

markg85 Wrote:edit:://
this mod is also looking interesting
o and this bandwidth module is also looking very interesting now you don`t even have yo log the traffic (unless you want traffic graphics) becuase with this module you can set bandwidth limitations. even things like: files larger than 500kb are downloaded with speeds not faster than 100kb/s

This mod is very interesting...could be an idea also to implement something that works with netfilter/iptables (or pf on the *bsd), like to build some queues for the virtual hosts...somebody wants more bandwith? We can put it in a different queue.
http://trekweb.com/~jasonb/articles/traf...arios.html

Pier