Author Topic: Failure with special character  (Read 865 times)

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 576
    • View Profile
Failure with special character
« on: March 02, 2023, 06:25:01 PM »
We have an issue when loading a name field.

 ARS1127E Row 1:  The string "JUAN CARLOSÑORTEGA                                               " has a length of 83 and the field has a maximum length of 80

The special character added by what I am assuming the "Ã" is causing the field to spill into 83 characters. I've tried removing space, removing tab, etc.

the only fix that I was able to get working in a "timely" matter - is recreate the AG, and for the field info:

String: Case(Mixed)
Type: Variable

IBM suggested that - But I am wondering if anyone knows of a way to do it at the application level so I don't have to recreate an application group.

#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Darrell Bryant

  • Full Member
  • ***
  • Posts: 104
  • Sed fugit interea fugit inreparabile tempus-Virgil
    • View Profile
Re: Failure with special character
« Reply #1 on: March 03, 2023, 06:10:41 AM »
If you have a test environment, you might experiment with the ARS_USE_OLD_STYLE_CASE parameter in the ARS.CFG file.

Use the following if upper casing expands the number of characters, for example the German Ezett (ß) expands to SS with the new style (ICU) upper casing
In the ARS.CFG:
# Use old style upper casing to prevent character expansion
#
ARS_USE_OLD_STYLE_CASE=1

When value of ARS_USE_OLD_STYLE_CASE is:

(0) - then we do all upper/lower casing in UTF-16 (no change)
(1) - then we do all upper/lower casing in OS level codepage we are running under (no change)
this was really added just to prevent the german sharp s (ß) character not expand to multiple bytes (which it does in UTF-8)
(2) - we now upper/lower case in UTF-16 (however we specifically prevent ß from being done).
We then convert this back to the native codepage (of the data) - then compare to the original string that was provided to us.
If we find any characters that are not found (e.g. ICU will change the char to be a substitution char),
we will put the original char back into the upper/lower case string. If for any reason this fails we will
default as if the ARS_USE_OLD_STYLE_CASE=1 for that string.
#IBMi #iSeries #PDF #XML #400 Indexer #ASM

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 576
    • View Profile
Re: Failure with special character
« Reply #2 on: March 03, 2023, 12:59:41 PM »
If you have a test environment, you might experiment with the ARS_USE_OLD_STYLE_CASE parameter in the ARS.CFG file.

Use the following if upper casing expands the number of characters, for example the German Ezett (ß) expands to SS with the new style (ICU) upper casing
In the ARS.CFG:
# Use old style upper casing to prevent character expansion
#
ARS_USE_OLD_STYLE_CASE=1

When value of ARS_USE_OLD_STYLE_CASE is:

(0) - then we do all upper/lower casing in UTF-16 (no change)
(1) - then we do all upper/lower casing in OS level codepage we are running under (no change)
this was really added just to prevent the german sharp s (ß) character not expand to multiple bytes (which it does in UTF-8)
(2) - we now upper/lower case in UTF-16 (however we specifically prevent ß from being done).
We then convert this back to the native codepage (of the data) - then compare to the original string that was provided to us.
If we find any characters that are not found (e.g. ICU will change the char to be a substitution char),
we will put the original char back into the upper/lower case string. If for any reason this fails we will
default as if the ARS_USE_OLD_STYLE_CASE=1 for that string.

Yup- I saw that as well. Do you know if that requires a recycle of ARSSOCKD? I only ask because we are on Z/OS and we'd have to do some coordination.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Darrell Bryant

  • Full Member
  • ***
  • Posts: 104
  • Sed fugit interea fugit inreparabile tempus-Virgil
    • View Profile
Re: Failure with special character
« Reply #3 on: March 03, 2023, 02:11:56 PM »
Any change to ARS.CFG requires a restart of arssockd.
#IBMi #iSeries #PDF #XML #400 Indexer #ASM

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 576
    • View Profile
Re: Failure with special character
« Reply #4 on: March 08, 2023, 11:05:04 AM »
Thanks, wasnt sure if it was the same on the mainframe side
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING