domain_traffic table growing huge

[sci2tech]

kilburn Wrote:Have you done anything on this front? I'm writing "my own vlogger" as a coding exercice, but don't really know if it's performance (in-memory size, cpu consumption) will be suitable for high load webservers.

Until sunday i don`t have time to do tests. But when i`ll finish i`ll put the conclusion here. I also do some test wit mod_sql. So far seems the easiest solution, but don`t know how a heavy loaded server will react. Sunday i`ll try to test both variants.

[Tseng]

kilburn Wrote:I'm trying to find a suitable solution for real-time curstomer logs because the "best practice" in php is to disable error displaying on production servers. Now the use case:

- user.tld uploads his website to our server
- user.tld loads his website using a browser
- user.tld gets a nice all-blank page
- user.tld can't see any error in his error log (he has to wait for the next rotation)
- user.tld calls the support center and gets angry because having to wait one hour between every fixed error is unacceptable for him

Two ways from here:

1. (Bad support) tells him that this is a server issue, nothing can be done about it and user.tld gets another hosting provider.
2. (Good support) tells him to use the display_errors directive, wich partly solves the issue (he will be able to see runtime errors, but not startup ones) and user.tld may or may not get another provider.

By the way, why won't you use vlogger? Any advice against it? (I haven't tried it so there may be drawbacks I don't know!).

sci2tech Wrote:... For me i`ll try a php aproach, but it will be nice to have a solution integrated in ispcp

Have you done anything on this front? I'm writing "my own vlogger" as a coding exercice, but don't really know if it's performance (in-memory size, cpu consumption) will be suitable for high load webservers.

Shouldn´t it be up to the developer/user to decide, if he/she want´s to display errors? The admin just has to be sure, that the user code can't break out and gain more rights than granted. Safe_Mode, suPHP, suExec ... ?

[kilburn]

The 'display_startup_errors' and some other directives can only be set using apache's config (mod_php) or the vhost's php.ini (fastcgi) so it isn't easy to let the user change them. To allow this we would have to modify the domain manager backend, probably alter the ispcp's database structure, etc. so it probably wouldn't make it to release 1.0.

Also too often users are lazy or non-experienced and simply keep the server's default, so it's important that the values we choose for them are sane ones.

[ispcomm]

Hmmm... there is not much information about php errors in the apache log files (mod_php). There're even less in suphp logs. I'm not sure about fastcgi but I guess the same applies.

I think the best solution is to force a user to use the php hooks (http://www.php.net/errorfunc) if required (for debugging on remote server).

Perhaps an easy (and dangerous) solution is to let the user upload a custom php.ini, but then we'll need to force some directives (like url extensions to fopen and include, which are quite used by php botnets, open_basedir etc).

Another solution would be to provide a (paid) debugging option where our vlogger(or what be) will be writing logs immediately to the user directory (and these could be picked up by an ispcp ajax tailer in the /tools/ directory). So you could open one browser for the php and another browser would be used for the ajax tailer (which would refresh on say 1-2 second intervals).

Being paid for, the user will probably disable it when he doesn't need it and there won't be so many open files on the server as now.

I'm just brainstorming some ideas ... feel free to comment.

ispcomm.

[kilburn]

In the php.ini file you can read:

Quote:; Print out errors (as a part of the output). For production web sites,
; you're strongly encouraged to turn this feature off, and use error logging
; instead (see below). Keeping display_errors enabled on a production web site
; may reveal security information to end users, such as file paths on your Web
; server, your database schema or other information.
display_errors = On

; Even when display_errors is on, errors that occur during PHP's startup
; sequence are not displayed. It's strongly recommended to keep
; display_startup_errors off, except for when debugging.
display_startup_errors = Off

; Log errors into a log file (server-specific log, stderr, or error_log (below))
; As stated above, you're strongly advised to use error logging in place of
; error displaying on production web sites.
log_errors = Off

The values above are from the default php.ini created for hosted domains (fcgi) and the default php.ini installed by the libapache2-mod-php package. IMHO we should disable the display_errors directive and enable log_errors as it is strongly encouraged. There will be much more php error info in the logs then

ispcomm Wrote:I think the best solution is to forse a user to use the php hooks (http://www.php.net/errorfunc) if required (for debugging on remote server).

The problem with this approach is that startup errors cannot be "hooked", because they don't allow the script to run, and the hooking is (not) done during it's execution. One example of such errors is a missing require (trivial to fix if you can see the error, nearly impossible if you don't)...

[ispcomm]

So, isn't the solution to keep display_startup_errors = on ?
During site "creation" this errors will be shown to the user.

When the site is operational there won't be any of them (i.e.... includes will be all in place).

This would be the least resistance path for ispcp imho. Perhaps not the best or finest solution, but for 1.0 release it might be enough.

ispcomm