Author Topic: ARS_ORIGINAL_CODEPAGE  (Read 1829 times)

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
ARS_ORIGINAL_CODEPAGE
« on: April 03, 2019, 04:28:45 AM »
Hello everybody,

I had a problem with our system, because of the code page, what is written in the database, what is displayed, etc...
And during all my test/java development in java and what was going on in prod, I found something very disturbing... and I wanted to know if you have the same experience, and what could be a solution...

Here is what I've done (AIX and Linux only)
- DB2 10.5, database created in ISO-8859-15 codepage, like we have.
- Install CMOD V8.4, create the tables/indexes, load some documents
- Upgrade to V8.5 or V9.5, doesn't matter.
- Add the ARS_ORIGINAL_CODEPAGE=923 as requested.

Here is what is strange, if I change the shell "locale" to UTF-8 when running arsdb -u for the upgrade process... then, I am requested to change ARS_ORIGINAL_CODEPAGE to the value 1208.
If I use the normal ISO-8859-15 code page for the shell, then ARS_ORIGINAL_CODEPAGE should be 923.
I play around to use other code (that was installed in my AIX/Linux) and guess what... the value of ARS_ORIGINAL_CODEPAGE is always dependant on the locale of the shell.

This has cause me a shock, and after discovering that, I could finally solve a bug I had for many years, that I could never reproduced. When I have my dev environment with the same shell locale as in production, then I could reproduce the problem and solve it.

This brings in my mind many questions... why does CMOD relies on the shell to determine the codepage to use?
This is something that can change at any time, for any reasons. Some will put UTF-8, some other to C, so other to something else.
Maybe you migrate from one server to another, and the old one was per default in ISO-8859-1 and the new one is per default configured with UTF-8...
Then Boom, you server will not react as expected, you have strange chars in your clients instead of your üöéàèôû...

Am I dreaming and imagining things? If that is the case, it means that we must be really careful on how to start CMOD, that it use the correct codepage every time, and that the combination DB codepage and shell codepage that starts "arssockd/arsobjd" must be in sync at all time.

What are you thought on that??
Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: ARS_ORIGINAL_CODEPAGE
« Reply #1 on: April 03, 2019, 08:02:55 AM »
My only thought is...  "Thank you for warning me."  :D

Yes, the codepage should probably be defined in the ars.ini or ars.cfg file -- or explicitly set in the CMOD instance owner's directory at install time, with instructions on NOT changing it.

And now that you mention it, I think I may have been bitten by this problem before, but my solution was to force everything to move to Unicode.  :)

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

Bob Shipman

  • Newbie
  • *
  • Posts: 1
    • View Profile
Re: ARS_ORIGINAL_CODEPAGE
« Reply #2 on: October 08, 2020, 05:33:16 AM »
I wonder if anyone has any thoughts - after a db2move from AIX to Linux - all is working but I'm getting db2 truncation errors when I search in the client.
CLI0002W - SQLSTATE 01004
Nothing in db2diag - so I think it related to code page issues.
I checked the AIX system and the database code page was "CODESET ISO8859-1 TERRITORY en_US", so I created the database on the Linux box with the same values and loaded the data.
I set ARS_ORIGINAL_CODEPAGE=923 in the ars.cfg - duplicating from the AIX system value.
The locale in my bash session on the Linux box is UTF-8
Any thoughts would be appreciated?