2007/05/07
IRC [12:49] <dekoy> hi guys
IRC [12:49] <dekoy> I'm looking for documentation for UTF-8/ Charset and oracle
IRC [12:50] <dekoy> can anybody point me to on how to output diff charset in aolserver (im pulling data from an UTF-8 enabled oracle server)
IRC [12:51] <dekoy> Hi Dossy
IRC [12:51] <Dossy> Hmm.
IRC [12:51] <Dossy> Are you always/only going to be emitting UTF-8?
IRC [12:52] <dekoy> no it's going to be a mixed
IRC [12:52] <Dossy> Look in the sample config .tcl for "OutputCharset"
IRC [12:52] <Dossy> mixed? hmm--that might be problematic
IRC [12:52] <Dossy> better charset handling support is something I really want to get into 4.5.x or for 5.0
IRC [12:53] <dekoy> from what I read too It might be a lot of work (rob mayoff's site) but I'm just trying to ouput German and spanish right now
IRC [12:54] <Dossy> and you don't want to do that in utf-8?
IRC [12:54] <Dossy> you'd rather do iso8859-x?
IRC [12:54] <dekoy> i did that but still getting the wrong output
IRC [12:54] <dekoy> that's where my confusion is
IRC [12:55] <dekoy> is my problem where aolserver does conversion ?!? when it's pulling data from oracle or it's just a matter or setting charset when i do ns_return or both
IRC [12:56] <Dossy> Not exactly sure--you'd have to add some debugging to see where it's getting mangled.
IRC [12:57] <dekoy> Thanks Dossy let me play with it some more
IRC [12:58] <Dossy> yeah, if you can point to a specific place where it's getting transcoded wrong or mangled, I can probably suggest some things to try.
IRC [12:59] <dekoy> thank you i'll add some Notice's
IRC [13:56] *** Hubertus joined the chat.
IRC [13:56] <Hubertus> hello
IRC [13:56] <Hubertus> ;)
IRC [14:01] <dekoy> Dossy do I need to set the aolserver NLS_LANG variable to something ?
IRC [14:01] <dekoy> hello
IRC [14:03] <dekoy> brb
IRC [14:05] <Dossy> dekoy: NLS_LANG is more for Oracle, not necessarily AOLserver.
IRC [14:05] <Dossy> hi, Hubertus.
IRC [14:06] <Hubertus> aolserver is fastes than lighttpd ?
IRC [14:06] <Hubertus> or better working ?
IRC [14:19] <Dossy> Depends on the workload/task.
IRC [14:39] <dekoy> Im getting ? like "Ich mu? wegen meiner Harnprobleme.. "
IRC [14:42] <Dossy> dekoy: from ns_db?
IRC [14:43] <dekoy> yep
IRC [14:55] *** cacrus joined the chat.
IRC [15:21] <Dossy> Hmm. Can you verify the data is correct in Oracle through SQL*Plus
IRC [15:24] <dekoy> Kreuz in ein Kästchen jeder Gruppe machen. Wählen Si
IRC [15:24] <dekoy> yep it got it right in sqlplus
IRC [15:25] <dekoy> according to RMayoff's site if the database is UTF-8 http://dqd.com/~mayoff/encoding-doc.html#db aolserver/tcl gets it right unmodified
IRC [15:33] <Dossy> Do you have the control port (nscp) configured?
IRC [15:33] <Dossy> or, can you create a quick test page--I want to know what [encoding system] returns.
IRC [15:33] <Dossy> When you run SQL*Plus, what is NLS_LANG set to?
IRC [15:34] <dekoy> sqlplus is NLS_LANG=AMERICAN_AMERICA.UTF-8
IRC [15:34] <dekoy> i don't nscp configured
IRC [15:39] <CIA-25> 03michael_andrews * 10aolserver/tcl/packages.tcl:
IRC [15:39] <CIA-25> Added packages.tcl:
IRC [15:39] <CIA-25> 1) Reads config to set lib paths.
IRC [15:39] <CIA-25> 2) Reads config to load packages using ns_ictl.
IRC [15:39] <CIA-25> 3) Runs nsInit procs.
IRC [15:39] <CIA-25> 4) Runs nsPostInit procs.
IRC [15:47] <dekoy> is this what you need ? Accept-Charset: ISO-8859-1,utf-8;
IRC [15:47] <dekoy> what's the variable for encoding system ?
IRC [15:53] <Dossy> no, I need what Tcl thinks its encoding is. That's returned from the Tcl expression [encoding system]
IRC [15:53] <Dossy> like, on one of my nsd's:
IRC [15:53] <Dossy> dossy.org:nscp 1> encoding system
IRC [15:53] <Dossy> iso8859-1
IRC [15:56] <dekoy> % encoding system
IRC [15:56] <dekoy> utf-8
IRC [15:56] <dekoy> %
IRC [15:57] <Dossy> that's from a tclsh, right?
IRC [15:57] <Dossy> not from the actual nsd
IRC [15:58] <dekoy> yep
IRC [15:58] <dekoy> i can't restart aolserver to get nscp working production server :)
IRC [16:04] <Dossy> ok, create a test .adp
IRC [16:04] <Dossy> foo.adp, <%= encoding system %> ... hit it in a browser
IRC [16:04] <Dossy> you don't have a dev server?
IRC [16:05] <dekoy> we do just got a hold on a dev server
IRC [16:07] <dekoy> <%=[ encoding system] %> return s iso8859-1
IRC [16:07] <dekoy> same thing in production
IRC [16:14] <cacrus> i just want to add something here , although it can be compoletely out of topic . But we have all our conttents and database utf8 .... to make aolserver respond everything utf8 , i always edit nsd/encoding.c file and add utf-8 values in a certain secion .
IRC [16:14] <cacrus> otherwise it just does not work for us .
IRC [16:14] <cacrus> if u wanna know where i add let me know , otherwise i will assume i am way off topic :)
IRC [16:15] <dekoy> im all ears
IRC [16:15] <dekoy> :)
IRC [16:15] <cacrus> ok wait let me read that file
IRC [16:15] <dekoy> thanks
IRC [16:16] <Dossy> what does <%= $env(NLS_LANG) %> sya?
IRC [16:16] <Dossy> er, say?
IRC [16:17] <Dossy> Darn, I wish I had my old AOLserver configs for Oracle...
IRC [16:18] <cacrus> OK these are the changes we do in nsd/encoding.c
IRC [16:18] <cacrus> static struct {
IRC [16:18] <cacrus> char *extension;
IRC [16:18] <cacrus> char *name;
IRC [16:18] <cacrus> } builtinExt[] = {
IRC [16:18] <cacrus> {".txt", "utf-8"},
IRC [16:18] <cacrus> {".htm", "utf-8"},
IRC [16:18] <cacrus> {".html", "utf-8"},
IRC [16:18] <cacrus> {".adp", "utf-8"},
IRC [16:18] <cacrus> {NULL, NULL}
IRC [16:19] <cacrus> };
IRC [16:19] <cacrus> and
IRC [16:19] <cacrus> on aolserver 4.0.10 its line 796
IRC [16:19] <cacrus> and in builtinChar block we also add
IRC [16:20] <cacrus> { "utf-8", "utf-8" }, line 161
IRC [16:20] <cacrus> btw last line number was wrong its 84
IRC [16:21] <cacrus> Dossy cna i help about oracle aolserver config ?
IRC [16:22] <dekoy> NLS_LANG is not set in production dev server has AMERICAN_AMERICA.UTF-8
IRC [16:23] <Dossy> cacrus: yeah, tell dekoy what env vars need to be set in the aolserver startup script so that the TNS client is config'ed right
IRC [16:23] <Dossy> dekoy: yeah, that sounds right ... American_America.UTF-8
IRC [16:23] <Dossy> although, you're not dealing with ISO-8859-1 are you.
IRC [16:23] <dekoy> tried it too
IRC [16:24] <dekoy> same output the special characters are being changed to a ?
IRC [16:24] <dekoy> cacrus do you have anything special in your startup script with aolserver ?
IRC [16:25] <cacrus> we always set NS_LANG to same as yours
IRC [16:26] <Dossy> can you create a test page?
IRC [16:26] <Dossy> a simple .adp that just does a ns_db query and immediately outputs the results with ns_adp_puts
IRC [16:26] <Dossy> without doing anything to the response (like regexp, etc.)
IRC [16:26] <Dossy> also, what is your OutputCharset set to?
IRC [16:26] <Dossy> in your nsd config .tcl?
IRC [16:27] <dekoy> cacrus I think the changes in encoding.c can be done in nsd.tcl on the section ns/mimetypes - ns_param .html "text/html; charset=utf8" I think
IRC [16:27] <cacrus> yes i know that
IRC [16:27] <cacrus> but that does not work
IRC [16:27] <cacrus> we have that too in the nsd.tcl
IRC [16:27] <dekoy> oh
IRC [16:27] <cacrus> but we always had issue until we compile aolserver with the cyhanges i mentioned
IRC [16:27] <dekoy> ns_param URLCharset iso8859-1
IRC [16:27] <dekoy> ns_param OutputCharset iso8859-1
IRC [16:27] <cacrus> yes we have played with all of that
IRC [16:27] <dekoy> oh hrmm
IRC [16:28] <cacrus> we also define btw LANG , so that our files get written properly on filesystem
IRC [16:28] <dekoy> ill give the recompile a try on the dev server thank you very much
IRC [16:28] <cacrus> LANG=en_US.UTF-8 Other than than i have export ORA_NLS33=$ORACLE_HOME/ocommon/nls/admin/data
IRC [16:28] <dekoy> so your db is UTF-8 (oracle 10g too ?
IRC [16:29] <cacrus> yes our database is utf-8
IRC [16:29] <dekoy> okay ill try that recompile on the dev server
IRC [16:29] <cacrus> i can send you my encoding.c if u want
IRC [16:29] <Dossy> dekoy: you have to set those to utf-8
IRC [16:29] <cacrus> just put it there and try to compile
IRC [16:30] <Dossy> not iso8859-1
IRC [16:30] <dekoy> please can you email it to me to likot@yahoo.com
IRC [16:30] <dekoy> okay dossy let me try that ( i can't remember if i tried that before)
IRC [16:30] <Dossy> OK.
IRC [16:32] <dekoy> hrmm same thing set NLS_LANG and changed to utf-8
IRC [16:33] <Dossy> create the simple test ADP page
IRC [16:33] <Dossy> also, when you change the configs, you have to restart the nsd
IRC [16:34] <dekoy> idid in the aolserver
IRC [16:35] <Dossy> you made the changes, restarted aolserver, and created a small test page and it's still not working?
IRC [16:36] <cacrus> dekoy: I have mailed you encoding.c
IRC [16:38] <dekoy> thanks
IRC [16:38] <dekoy> DOssy yep
IRC [16:39] <dekoy> thank you guys for your help im being pulled in a meeting.
IRC [16:39] <dekoy> if you don't mind i'll come back and if you guys are not busy ask you some questions again
IRC [16:39] <dekoy> thank you
IRC [16:39] <dekoy> .
IRC [16:43] *** cacrus parted the chat.
IRC [16:57] <Dossy> sure thing. if we don't answer right away, don't be surprised :)
IRC [18:25] *** dekoy parted the chat.