ispCP - Board - Support
how do you realize the network traffic controll stuff? - Printable Version

+- ispCP - Board - Support (http://www.isp-control.net/forum)
+-- Forum: ispCP Omega Development Area (/forum-1.html)
+--- Forum: General discussion (/forum-11.html)
+--- Thread: how do you realize the network traffic controll stuff? (/thread-114.html)

Pages: 1 2

how do you realize the network traffic controll stuff? - markg85 - 01-25-2007 05:13 AM

Hey,

i`m just wondering.. how do you grab all the network traffic and and grab out the parts you need to display how much traffic a certain (sub)somain has been used?

or.. just how do you do this stuff?
i`m just curious Smile

Thanx in favor.

RE: how do you realize the network traffic controll stuff? - ephigenie - 01-25-2007 05:34 AM

Traffic calculation is done through apache logs for http per domain,
ftp traffic is logged in mysql (proftpd) per user (then added)

and so on.

RE: how do you realize the network traffic controll stuff? - markg85 - 01-25-2007 05:47 AM

than you have a damn hard log analyzer.. Tongue

this is a part of my apache access log file:
[quote]
127.0.0.1 - - [10/Dec/2006:16:20:06 +0100] "GET / HTTP/1.1" 200 19068 "-" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1) Gecko/20061027 Firefox/2.0"
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/style.css HTTP/1.1" 200 2046 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.8.1) Gec$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/homepage.gif HTTP/1.1" 200 2226 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/search.gif HTTP/1.1" 200 1990 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/oodomimagerollover.js HTTP/1.1" 200 3923 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US; r$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/exed-banner.png HTTP/1.1" 200 22956 "http://pafiledb353.mage.nl/" "Mozilla/5.0 (X11; U; Linux x86_64; en-US;$
127.0.0.1 - - [10/Dec/2006:16:20:07 +0100] "GET /skins/default/images/row_bg.gif HTTP/1.1" 200 218 "http://pafiledb353.mage.nl/skins/default/style.css" "Mozilla/5.0 (X11; U; Linu$
[/qoute]

how do you split all that stuff? on spaces? or better.. what`s the file where this is done? than i can have a look for myself Smile

RE: how do you realize the network traffic controll stuff? - ephigenie - 01-25-2007 05:53 AM

have a look at all that /var/log/apache2/<domain.tld>-traf.log files Wink

RE: how do you realize the network traffic controll stuff? - markg85 - 01-25-2007 06:22 AM

http://httpd.apache.org/docs/2.2/mod/mod_log_config.html

handy Big Grin i didn`t know that
now i`m gonna try and find out that proftpd stuff Tongue

you guys surely did a smart way with that traffic stuff.. i realy would have thought that all the network traffic was analyzed and splitted to the right domains.. but this is alot better Smile

RE: how do you realize the network traffic controll stuff? - markg85 - 01-25-2007 06:57 AM

o.. just found this

http://bitbrook.de/software/mod_log_mysql/
and:
http://www.outoforder.cc/projects/apache/mod_log_sql/ (seems to be better)


with that you can store the log in mysql.. perhaps that`s a better way of doing it?
perhaps this can be used in the omega project Smile or even in isp control panel.

RE: how do you realize the network traffic controll stuff? - RatS - 01-25-2007 08:10 AM

if you are think it's a good idea, implement it!!

There are NO limits.

RE: how do you realize the network traffic controll stuff? - ephigenie - 01-25-2007 09:37 AM

mod_log_mysql has its limits on busy hosts. But a good solution perhaps later on.
I've been using it about 2,5 years now on one of my servers.
But there're a lots of bugs in mod_log_sql and it is not ported cleanly to apache2.2.

(i speak of the mod_log_sql module) the developers promised one day to support the new ap2db - layer (apache2.2 can handle db - connections himself and keep a pool for modules like php and so on (only mod_php) )

The Traffic calculation can maybe one day be replaced with libapache-mod-acct-mysql if this one gets continued (at the moment only apache1 is support afaik)

in ISPCP we've to go even different ways, because we like to make the whole thing multi-server aware and aware of directing certain services to certain machines - such as a dedicated mail-server and so on..

RE: how do you realize the network traffic controll stuff? - markg85 - 01-26-2007 05:11 AM

o wow that`s cool Smile
when i get some free time and when there is something of that ispcp released (builds/beta`s) i will probably start testing out some good stuff Smile (your looking at a future team member here Wink)

edit:://
this mod is also looking interesting
o and this bandwidth module is also looking very interesting Smile now you don`t even have yo log the traffic (unless you want traffic graphics) becuase with this module you can set bandwidth limitations. even things like: files larger than 500kb are downloaded with speeds not faster than 100kb/s Smile

RE: how do you realize the network traffic controll stuff? - pierg75 - 01-26-2007 09:09 PM

markg85 Wrote:edit:://
this mod is also looking interesting
o and this bandwidth module is also looking very interesting Smile now you don`t even have yo log the traffic (unless you want traffic graphics) becuase with this module you can set bandwidth limitations. even things like: files larger than 500kb are downloaded with speeds not faster than 100kb/s Smile

This mod is very interesting...could be an idea also to implement something that works with netfilter/iptables (or pf on the *bsd), like to build some queues for the virtual hosts...somebody wants more bandwith? We can put it in a different queue.
http://trekweb.com/~jasonb/articles/traffic_shaping/scenarios.html

Pier