Author Topic: SFTP conversion  (Read 2972 times)

htees

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
SFTP conversion
« on: November 07, 2019, 01:04:28 PM »
Hi all; Our Company is currently converting our file transfer protocol from FTP to SFTP.  We are finding that pure text index files(used in the generic indexing process) are being re-formatted into one long line of text, rather than a simple text/index file with line feeds to separate each index field or value. 

We tested using a command to( ls /+mode=text) send over the files and they did send correctly, However we have been told by the FTP group that they cannot alter their tool/software and that we will need to come up with a solution. They are using something called ‘IBM Sterling Integrator [Java Based application]’ and they are stating that they cannot make changes toit.

So My question is has anyone else experienced this ? what measures did you take? Is creating some sort of pre-processor my only option?   

Thanks for any insight you can offer.   Harold Thomas 
Harold Thomas
Developer Advisor, Anthem BC/BS

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: SFTP conversion
« Reply #1 on: November 07, 2019, 01:15:31 PM »
It sounds like there's character conversion being done -- either FTP was fixing it, or SFTP is breaking it.  :D

I'm fairly certain that SFTP prefers binary mode, so either the file being produced wasn't compatible with your OS (and FTP was fixing it by transferring in ASCII) or it was, and transferring with SFTP is breaking it by changing the line ending characters.

You didn't mention which OS / CMOD version, but your server system administrator should know the best way to deal with this in your environment - it's been a remarkably common issue between different operating systems (UNIX / Linux / Mac / Windows / Mainframe) for decades.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

htees

  • Jr. Member
  • **
  • Posts: 17
    • View Profile
Re: SFTP conversion
« Reply #2 on: November 08, 2019, 12:16:37 PM »
Thanks Justin; We were PURPOSELY transferring in Ascii to make sure we could get the index files in plain text when sent to our Unix directory on the ZOS mainframe. These files are being loaded through the Generic indexing process on CMOD 9.5. If SFTP prefers Binary, what is a good way to get these files either as plain text into our Unix directory, or to process Binary index files?

Thanks HT
Harold Thomas
Developer Advisor, Anthem BC/BS

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: SFTP conversion
« Reply #3 on: November 09, 2019, 12:58:19 PM »
It's probably best to get the source system to produce the CMOD Generic Index File with the correct line endings for the destination system.

If not, there's a ton of ways to convert line endings when you're in a UNIX environment.

You didn't mention what the source system is, and I'm guessing you're using USS on z/OS.  If the source is Windows, you need to convert CRLF (carriage return+line feed) to NL (newline).

Your local (and likely bearded) UNIX guru should be able to help you do this in their preferred way.  I'm not sure which UNIX tools are available in USS.

-JD.

IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR