Author Topic: ARSLOAD best practices for failed and successful load  (Read 3307 times)

vjjdavid

  • Guest
ARSLOAD best practices for failed and successful load
« on: November 03, 2015, 11:55:41 AM »
I need to run ARSLOAD and need these two scenario to take place
     1- If load is successful, mark file with .DONE & move file to folder A after loading.
     2- If load failed mark file with .FAILED & move file to folder B after loading.
In case of any issue, we want to be able to have these file for possible reprocess.  How are you guys setup out there? Kindly share any suggestions you may have.

Justin Derrick

  • IBM Content Manager OnDemand Consultant
  • Administrator
  • Hero Member
  • *****
  • Posts: 2228
  • CMOD Guru for hire...
    • View Profile
    • Tenacious Consulting
Re: ARSLOAD best practices for failed and successful load
« Reply #1 on: November 03, 2015, 01:33:59 PM »
While arsload can't do this sort of thing on your behalf, you can write a 'wrapper' in shell script that calls CMOD.

In arsload, use the -n option (for "No-delete") and it will leave the files behind.  After arsload exits, you can move (and optionally compress) the data for safe-keeping.

-JD.
IBM CMOD Professional Services: http://TenaciousConsulting.com
Call:  +1-866-533-7742  or  eMail:  jd@justinderrick.com
IBM CMOD Wiki:  https://CMOD.wiki/
FREE IBM CMOD Education & Webinars:  https://CMOD.Training/

Interests: #AIX #Linux #Multiplatforms #DB2 #TSM #SP #Performance #Security #Audits #Customizing #Availability #HA #DR

Maciej Mieczakowski

  • Jr. Member
  • **
  • Posts: 37
    • View Profile
Re: ARSLOAD best practices for failed and successful load
« Reply #2 on: November 04, 2015, 01:40:41 AM »
Exactly as Justin says it can be easily solved with short shell script if you uses Linux/UNIX.

You can also use another approach without ARSLOAD usage

I do the same thing as you described, but with a day delay (as I have a complete list of files that have not been processed). So I do not even use arsload functionality but simply query systemlog and take the complete list of files that failed to load from previous day. I do not bother about files that have been successfully processed, all that matters from my perspective is data that failed to load as there is a need to check this out and reprocess if possible.

If you need a report of successfully processed data you can simply use 87 records from system log as well.

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: ARSLOAD best practices for failed and successful load
« Reply #3 on: November 04, 2015, 04:55:46 AM »
I always do the same way as Justin proposed.

The was of Maciej is also nice :-)

I will add a third one, just to give a bit more choices :-D
In fact it is an extension of Maciej method!
Instead of waiting X hours, you can by modify the "arslog" command to catch the message 87 (good) and 88 (not good) to handle the files as you want them.
So that way you get a automatic quick way to handle your files the way you want, as soon as they are finished processed.
Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 573
    • View Profile
Re: ARSLOAD best practices for failed and successful load
« Reply #4 on: November 04, 2015, 06:39:16 AM »
Easily done via shell script. Have a script that runs via cron or whatever automation tool of choice looking for files that end in .Failed, move them to a seperate directory that is not monitored and then spend a few minutes each day looking into why the files failed. In my experience its been garbage in garbage out, then keep track of via a spreadsheet.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

vjjdavid

  • Guest
Re: ARSLOAD best practices for failed and successful load
« Reply #5 on: November 04, 2015, 08:49:35 AM »
Thank you all for the may options.  We have a slight change in the process where we have data coming from Mainframe and BARR and sitting on a Windows server.   We are looking to rung ARSLOAD on the Windows server while the CMOD Server in on Linux.  We've decided to bypass the FTP process to move the files from Windows to Linux.

Does these options still apply?

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: ARSLOAD best practices for failed and successful load
« Reply #6 on: November 04, 2015, 09:57:46 AM »
these options are "generic", then it depends on your own skills :-)

I nearly need to use windows for CMOD... and I use quite exclusively Linux/Unix... Except from time to time to do some education on new CMOD people, and I do it on windows, and I also have a linux VM to show how to do it there :-)
so I can nearly do all the advises in Linux/Unix with my closed eyes!
If I needed to do it in Windows... then I would to open my eyes and look in Google :-D

Now, that said... if you want to use "arsload" in Windows, but your CMOD server is in Linux... then my option with the modifying of "arslog" won't work in your case. Because the arslog will be started on the CMOD server which is in Linux, and you are running arsload in Windows :-) therefore that won't be easy to run such cross environment scripts!!! so I won't advise it all all !!

Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML