Bad Character???

[DDP021]

Hi, we have an old report defined using OS390 indexer..Its been running for years...recently its been failing on an index (USER NAME) with the below error:

The string "SISSOKO, AØÏSSATA " has a length of 20 and the field has a maximum length of 18

I know its do to either the character that looks like an O with a line through (x'80') it OR the I with the 2 dots above it (x'57') because when we remove those 2 characters, the file loads.  We did attempt to remove each one individually and received the same results..But removing both allowed the file to load successfully...We even attempted to created a new definition using ACIF indexer with same results...Had anyone else ran into this kind of issue with special characters? Also, the INDEX is set as a STRING.

Thanks in advance for any input

Take care

Dave

[Ed_Arnold]

Hello Dave - I ran into this in 2016 on z. 

Here's some discussion I had on this:

 With 8.5 comes greater accuracy with handling of non-US characters.
 Typically, this is a symptom of the German "Sharp S" which when
 uppercased becomes two letters.
 
 Here's how you fix it:
 
 a) install PM82660/UK92465
 
 b) add the following to ars.cfg:
     ARS_USE_OLD_STYLE_CASE=1
 
 That's all that's needed to resolve the loading problem.
 
 However,
 
 c) all clients should be at 8.5.0.6 or higher to deal with the
 character being stored as a Sharp S when doing searches.
.....................................................................
 
Related APAR:
 
.....................................................................
 
APAR PM70770 / UK81470 Description
 
 Upper-casing the German s-sharp produces SS.  This expansion of
 the string caused the NULL that  terminated the string to be
 overlaid, causing subsequent string operations to behave
 incorrectly.
 
 PROBLEM CONCLUSION:
 ****************************************************************
 * PROBLEM DESCRIPTION: When running arsload with a applgrp     *
 *                      field that specifies upper-casing and   *
 *                      the field contains a German s-sharp     *
 *                      (U+00DF), the field can either be       *
 *                      truncated, or arsload can abend due to  *
 *                      heap  corruption.                       *
 ****************************************************************
 * RECOMMENDATION:                                              *
 ****************************************************************
 ARNADMP is changed to detect the expansion of the string due to
 the upper-case operation and not overlay the trailing NUL.
 Additionally, ARNADMP will detect if the expansion exceeds the
 size of the field and issue an error message.
 
..................................................................
 
This issue was reported on z and fixed on z.

But then along came the problem on AIX.

For indexing and loading on AIX you also add the following to ars.cfg:
 
     ARS_USE_OLD_STYLE_CASE=1


Just for completeness of this discussion, there was still a problem loading from AIX -> z.

A colleague weighed in with this:

I have been looking at how to fix this issue and trying to use ars.cfg /
ars.ini on your AIX machine may prove to be a bit tricky.
 
From my tests here on the AIX box you can also specify an environment
variable. Do this for the user(s) who run the arsload command:
 
ARS_USE_OLD_STYLE_CASE=1

The reply was

...it works!! ...
 
I put
 
export ARS_USE_OLD_STYLE_CASE=1
 
before the call of arsload

Ed

[DDP021]

Thanks Ed!...We are running Version 9.5.0.8

I will pass this info onto our systems group to see what they can possibly do.

Appreciate the info!!!!!

Take care

Dave

[Ed_Arnold]

Dave - I updated my post after your last post.

Just FYI to pick up the (minimal) changes.

Hope this helps!

Ed

[Justin Derrick]

This issue was also mentioned in one of the previous ODNews newsletters:  http://www.ibm.com/support/docview.wss?uid=swg27041378

-JD.