AOLserver Cookbook
A collection of questions, and code examples to answer them.
Add your questions here. Answers should be placed below the question. Place a horizontal line ("----") between questions. Please keep this text before the first question.
Limiting bandwidth
Is there a simple way to limit bandwidth per connection/ip???
- There's no simple way of doing it, i.e., no configuration option you can set. If I had to implement this, I'd use NSV's and ns_register_filter to keep track of bandwidth consumed by each client IP which can act as an all-or-nothing mechanism. However, what might be interesting to implement in the socket driver is a setting to control the rate at which bytes are written back to the socket, perhaps configurable per request, so that large HTTP responses don't soak up your upstream bandwidth. We should keep this in mind for a future version of AOLserver. -- Dossy 20:06, 4 November 2005 (EST)
One more question... How can I read output buffer inside function registered as postauth filter? And how can I stream it? ns_adp_puts cannot be used inside filters... ns_return would return all data at once. I assume that per connection limit would look like this:
proc ..... {
ns_adp_stream
set data [get output content buffer]
set data_length [output buffer size]
set block_size 4096
for {set i 0} {$i < $data_length} {incr i $block_size} {
send_to_client [string range $data [expr $block_size*$i] [expr $block_size*[expr$i+1]]
sleep 1
}
ns_returnok
return filter_return
}
I want to limit only one vhost so nssock limit would be too much;)
- First, you don't want to turn ns_adp_stream'ing on. Getting the size of the ADP output buffer can be done using ns_adp_tell and getting the contents with ns_adp_dump. I guess you could ns_adp_trunc the output buffer to clear it, then slowly write the contents back using ns_write. You'd also then need to look at ns_conn outputheaders to make sure that you wrote the proper HTTP response ... ah, but I don't know if you can get the HTTP response status code somehow. I still think doing it in nssock is probably the best place, even with vhosts -- I'd make it a preauth or postauth filter that sets the configuration for the response and then have nssock be responsible for ensuring the response is written at that rate. -- Dossy 07:24, 8 November 2005 (EST)
ns_adp_tell inside postauth filter:
Error: This function cannot be used outside of an ADP
what's more, i use ns_returnfile inside some adp code and if i use ns_adp_tell after it, it returns 0 (this adp only returns file, no adp_puts or write). Unfortunetly i think there is no other solution then implement it inside nssock or implement some tcl code to manipulate all output buffer.
Virtual server configuration
Is it possible to use wildcard '*' in domain name inside "ns/module/nssock/servers" section of aolserver4 config file? I would like to map all requests to *.somedomain.org to one virtual host... Something like this doesn't work:
ns_section "ns/module/nssock/servers" ns_param forum somedomain.org ns_param forum somedomain.org:80 ns_param forum *.somedomain.org ns_param forum *.somedomain.org:80 ns_param forum "*.somedomain.org" ns_param forum "*.somedomain.org:80"
-Paul Bukowski
- 2005aug15 Dossy: Unfortunately, no -- you currently can't use wildcards the way you want. Not yet, at least.
Why won't you get some code for mass virtual hosts from http://naviserver.sourceforge.net/ ?? -PB
- Right now, to map a request to its (virtual) server, there's an O(n) search being done, but it's a "cheap" exact string comparison. To implement glob-style wildcard support, it'll become a more expensive (but possibly still "cheap" by most people's standards) glob-match comparison. When you're serving "n" requests per second (where "n" is small) the difference in cost is probably immeasurable, but when "n" is large, the cost is additive and possibly significant. It might be good if someone could benchmark the two and see what the real cost is. -- Dossy 20:12, 4 November 2005 (EST)
Server-side Includes
Does anyone have some code to implement apache-compatible SSI (Server-side includes)? I have some old pages that use SSI that I'd like to move to my aolserver install but I don't have the time to rewrite them as ADPs.
- Dossy 20:20, 4 November 2005 (EST): I make no representations or warranties about the safety (!) or correctness of the code I'm about to share, but it might get folks started on a proper implementation in the future. In particular, I didn't implement any commands other than "include" and I even implemented it "wrong" (not according to the specification). If there's real interest in having a complete SSI implementation for AOLserver, I'd be willing to improve this code.
ns_log Notice "ssi: registering SSI ADP tags"
proc ssi_include_tag {params} {
set param_count [ns_set size $params]
for {set i 0} {$i < $param_count} {incr i} {
set ssi_var_[ns_set key $params $i] [ns_set value $params $i]
}
if [info exists ssi_var_file] {
if {[ns_adp_argc] > 1} {
set basedir [lindex [ns_adp_argv] 1]
} else {
set basedir [ns_adp_dir]
}
if [regexp {^/} $ssi_var_file] {
set filename $ssi_var_file
} else {
set filename $basedir/$ssi_var_file
}
return [ns_adp_parse -file $filename $basedir]
}
}
ns_register_adptag {!--#include} ssi_include_tag
ns_log Notice "ssi: registering SSI ADP tags done"
Is there a way to see only the errors (for example - 'variable' xxx not found? I seem to remember 2 logs - servername.log and servername-error.log? The version I have now - aolserver 3.2/ad1.2 only seems to give us servername.log - and all the error messages are mixed with the normal msgs - its very hard to debug! dannyl50
The default/example config file did not have entries for both logs. The two sections you need look like (in old style notation):
ns/parameters User=nsadmin Group=nsadmin ServerLog=/var/log/aolserver/servername-error.log PidFile=/var/log/aolserver/nspid.servername Home=/usr/bin/aolserver-3.3ad13
and
ns/server/servername/module/nslog File=/var/log/aolserver/emp.log LogCombined=On MaxBackup=3 RollFmt=%Y-%m-%d-%H:%M RollHour=0 RollOnSignal=On RollLog=On
Howdy! Can someone tell me, in general terms, how they have implemented nsperm? It looks cool, but I am using a plain old database with names and passwords, and it is very easy. I would like the allow/deny functionality, can I have that without the username/password part? -- Ian Harding
I've implemented nsperm to restrict access to specific URLs using HTTP authentication (the username/password dialog) and cookie-based (if they have the correct cookie, otherwise URL redirect them to a login page that may set the cookie). The latter is done using a registered preauth filter. I chose to use nsperm over a database since nsperm (presumably) manages the authentication information in memory better than repeatedly hitting the database with authentication requests, but there's no reason I couldn't have used a database to store username/password information. Does this answer your question? -- Dossy
Pretty much, although I started writing my app before I understood AOLServer, so I have a rather strange design that is good in some ways, bad in others. Keep in mind that this is a low activity app, no more than 20 concurrent users right now.
What I did was to create an ADP that has a big case statement. It calls functions to generate pages based on the 'task' value it is passed through POST. I have created a pretty complex app this way, not using ns_register_proc at all. I know now I should have. As I understand it, ns_register_proc lets you create a 'virtual' directory structure that actually just calls certain procedures based on the GET or POST requested URL. This is far more graceful than hitting my main adp which opens up the form data, looks at the 'task' value, and decides which procedures to call.
Given this lame structure, each time someone requests the main adp, I just check for a valid cookie and generate a login page if it's not there. After that, I have to set another cookie that is their 'access rights' which is then checked from within the procs that generate the pages. If I was doing it right, I could have registered a proc that would fire for all URLs in my virtual tree, checking your rights to the particular URL you reqested, and redirecting you if you are not allowed, right?
Thanks!!! -- Ian Harding
I probably would've created an ADP per task, and then registered a preauth filter that checked for the cookie and redirected the user to the login page (outside the directory that the preauth filter covers -- avoid funny redirection loops) if the cookie wasn't set properly. If you don't want to create a separate ADP per task, you could have used ns_register_proc to create what you describe as "virtual pages" and still have the preauth filter kind of "sit in front" and handle the autehntication, keeping that logic seperate from your actual page generation code. -- Dossy
I started with a separate adp for each task, but I thought it would get out of hand quickly. I do think I will re-work the thing to use ns_register_proc and even do away with the few adp I do have. It seems far easier to maintain. In the meantime, I have to re-write all my SQL Server procedures in PostgreSQL PL/Tcl. That's why I want the nsfreetds module so badly, it buys me a little time!
Thanks for all your work! -- Ian Harding
Speaking of nsfreetds, did you try nsfreetds-0.1pre? Any luck? Any problems? -- Dossy
I just now tried it. I installed freetds with no trouble, compiled nsfreetds, modified my nsd.tcl config file, added the datasource to interfaces, and all seems fine, except for that I always get "no access to pool "mypool" while executing "ns_db gethandle mypool" ..."
I must have missed something. I saw something in the README about setting $env(SYBASE) before starting nsd. Is this the same as the SYBASE=/usr/pkg/freetds I had to do even before the build of nsfreetds?
Thanks again... I feel like I am close.. -- Ian Harding
Yes, the $env(SYBASE) means setting the SYBASE environment variable. To be safe, you can try this:
$ cd /path/to/aolserver/install $ SYBASE=/usr/pkg/freetds; export SYBASE $ bin/nsd ...args...
Check log/server.log to make sure that the nsfreetds driver is actually loading. You should see a lines like this at start-up:
30/May/2001:13:42:3711838.1024-main- Notice: modload: loading '/home/aolserver-dev/bin/nsfreetds.so' 30/May/2001:13:42:3711838.1024-main- Notice: Ns_FreeTDS_DriverInit(freetds): Loaded Panoptic FreeTDS Driver v0.1-pre, built on May 21 2001 at 22:23:01.
Are you getting an error in the server log instead? -- Dossy
I don't get an error, it seems to load nicely. I can post or give you access to whatever files or output you would like to see, or an ssh login... -- Ian Harding
Another question: Global variables (nsv*). I am re-writing my app to use the ns_register_proc functionality versus my kludge where everything goes through an ADP that acts as a filter and traffic cop. I have figured out register filter and register proc, but seem not to be able to use ns_puts from within the procs. Is this right? Anyway, I had been using ns_adp_include in my filter/traffic cop page to put consistent headers/footers on my pages. I gather the more correct way (and only possible way with registered procs?) is to use global variables.
When do I initialize them? How do I use them? What are the caveats? W
Thanks, -- Ian Harding
Have you started by reading the NSV documentation? -- Dossy
Of course not 8^0 That would be too easy! I will... -- Ian Harding
OK, I read it. It's too easy. I use it for header, footer, and client side script html, which I stuff into html variable before returning it. I would like the filter option we talked about in the chat which would prepend or append html automatically, but this will do. -- Ian Harding
How hard would it be to add an HTTPS equivalent of Ns_FetchURL?
If you can build shared libraries on your target platform with OpenSSL, it probably wouldn't be too hard. The trick would be writing enough stub code that can be loaded via the Tcl "load" proc so that you could access the OpenSSL-based https client from your Tcl code. Is there a lot of demand for this? I could probably bang something out as an initial attempt. -- Dossy
Scott Goodwin's nsopenssl now contains client side https functionality.
Am I a brick, or does ns_register_proc just plain not work with more than one argument for the proc? I have tried the following code on both aolserver 3.2 running on NT4 and aolserver 3.4 on NetBSD 1.5. It fails to register the proc even once, if I try to register it with more than one argument anywhere else.
ns_register_proc GET /foo1 foo 1 ns_register_proc GET /foo2 foo 1 2 ns_register_proc GET /foo3 foo 1 2 3
proc foo { conn one {two 22} {three 333}} {
ns_return 200 text/html "one is $one, two is $two, three is $three"
}
This must be something simple. I have already posted to the aolserver list but the listserv seems to have gone mad.
What are you trying to do? It appears you want to pass constant parameters to your procedures. If they are constants, why pass them? Since ns_register_proc calls are generally in the global scope (and sourced when nsd starts) even a parameter that appears to be a variable probably can/will not vary.
Presumably you need to get information from one page to another. That means formvalues unless your page is stateful (usually a bad idea). Check out the ns_conn form command to get yourself started. -- Jason Kane
OK, I'm using Windows 2000 SP2 and 3.4.1, and trying in vain to get Perl CGIs (legacy) to work. The scripts work outside of Aolserver, and work on AOLserver on Linux, but when I try inside the script to GET from myself (eg, GET http://localhost:8000/foo?bar=bas), I'm told "unknown protocol TCP", which is quite aggravating.
Does anyone have any experience administering AOLServer on Windows?
Thanks in advance!
-Bill
This is interesting, wy do you let me change your page? I'm trying to find an example of how to get my data using:
set formdata ns_conn form $conn
-Jason
Because it's a wiki! A proc that gets posted variables includes the line you wrote there... then when you want to actually get the values, you can use something like
set myvar ns_set get $formdata myvar
where myvar is the name of the variable. There is also ns_queryget which looks like
set myvar [[[ns_queryget]] myvar defaultval]
This is handy because if there is no value, it will assign a default value.
Check out aolserver.com, they have extensive (although sometimes misleading) docs. If something the docs says should work doesn't, ask. It may be out of date.
I don't know where else to share my code fragments, so here seems like as good a place as any. This one implements user public_html directories. It's based on someone else's stuff (I forget whose tho). It should really do something with unknown mime types, and should do a better job with errors like unknown users. It does honor directoryFile and adp maps in the config file.
It works by registering a procedure to /users that interprets the username and finds their directory. ns_register_proc doesn't allow wildcards other than in the last path element, so /~* wouldn't work. However that's not a problem for a filter, so there's a filter to take your /~user url and redirect it to /users/user
-JR
proc homedir { conn ctx } {
set url ns_conn url $conn
set user {}
set file {}
regexp {^/users/(^/*)(/.*)?$} $url dummy user file
# ns_returnnotice 200 "url is $url user is $user, file is $file"
set userfile getUserFile $user $file
set type ns_guesstype $userfile
if {[[string match "*[ns_config ns/server/[ns_info server]]/adp map]" $userfile]} {
# ns_log notice "adp parsing $userfile"
set cwd pwd
cd file dirname $userfile
# ns_return 200 text/html ns_adp_parse -file $userfile
set result [[catch {ns_adp_parse -file $userfile} error]]
if {$result} {
ns_return 200 text/html $error
} else {
ns_return 200 text/html $error
}
cd $cwd
} else {
switch $type {
"*/*" {
}
default {
ns_returnfile 200 $type $userfile
}
}
}
if {[[catch {set fp [open $userfile]]}]} {
ns_returnnotfound
return
}
}
proc getUserFile {user path} {
foreach tail [[split ,[ns_config ns/server/[ns_info server]] directoryfile] ,] {
set userfile "glob ~$user/public_html$path$tail"
if {file exists $userfile && !file isdirectory $userfile} {
return $userfile
}
}
ns_returnnotfound
# break
}
ns_register_proc GET /users/ homedir
ns_register_proc POST /users/ homedir
ns_register_proc HEAD /users/ homedir
proc userRedir {conn arg why} {
set url ns_conn url
regsub {^/~(.*)} $url {/users/\1} redir
ns_log notice "url is $url, redir is $redir"
ns_returnredirect $redir
return filter_break
}
ns_register_filter preauth GET /~* userRedir
Hi!
Does anybody know a graphing module for Aolserver/TCL, with e.g. bargraphs, linegraphs,... ?
-wiwo
Yes: nsgds, nschartdir.
Can I detect if the user pushes the Stop button in their browser in my TCL/ADP page?
Yes, although it's a kludge. ns_write will return zero if it cannot write because the connection has been lost. However, ns_write is not stupid enough to try writing when you give it nothing to write, and so you must send something to the client. To prevent this being rendered use HTML comments, e.g.
if {ns_write "" == 0} {
# user hit the stop button, browser page was closed, or connection to client was lost.
}
