Author Topic: Quantifying extract & reload  (Read 1890 times)

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 573
    • View Profile
Quantifying extract & reload
« on: February 26, 2019, 12:13:34 PM »
Hi all,

Wondering if anyone cares to share their insight, tips, and past experience on this.

As some previous posts state, I am looking into extracting data from an AIX system, using TSM/Centera, into a new system using Linux/ICOS. I am trying to put an actual timeframe on how long it is going to take to extract from AIX, and reload into Linux. I have the extract/reload process down mostly, but I am not sure where to start as far as coming up with solid dates.

There is a total of around 7TB of data to extract, totaling 200 Million Documents across 32 application groups. The totals of data in our app groups range from a few hundred MB, to 4.5 TB.

I understand there are SEVERAL variables to consider as far as this entire process goes, but does anyone have any suggestion as to how I can use our logs to actually put some realistic numbers to this.

Thanks in advance.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING

Alessandro Perucchi

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 1002
    • View Profile
Re: Quantifying extract & reload
« Reply #1 on: February 28, 2019, 10:15:32 AM »
Hi all,

Wondering if anyone cares to share their insight, tips, and past experience on this.

As some previous posts state, I am looking into extracting data from an AIX system, using TSM/Centera, into a new system using Linux/ICOS. I am trying to put an actual timeframe on how long it is going to take to extract from AIX, and reload into Linux. I have the extract/reload process down mostly, but I am not sure where to start as far as coming up with solid dates.

There is a total of around 7TB of data to extract, totaling 200 Million Documents across 32 application groups. The totals of data in our app groups range from a few hundred MB, to 4.5 TB.

I understand there are SEVERAL variables to consider as far as this entire process goes, but does anyone have any suggestion as to how I can use our logs to actually put some realistic numbers to this.

Thanks in advance.

The only way to know for sure is to take an AG, and look how well it exports, and then how fast it imports.
From my experience, if you have an AG with lots of small documents, with lots of load ids, then it might take an awful long time, compared to an AG with big objects inside.

Since I've nearly never do an "arsdoc get" / "arsload" for such migration, I have very little details to give you.
Alessandro Perucchi

#Install #Migrations #Conversion #Educate #Repair #Upgrade #Migrate #Enhance #Optimize #AIX #Linux #Multiplatforms #DB2 #Windows #Oracle #TSM #Tivoli #Performance #Audits #Customizing #Availability #HA #DR #JavaApi #ContentNavigator #ICN #WEBi #ODWEK #Services #PDF #AFP #XML

jsquizz

  • Global Moderator
  • Hero Member
  • *****
  • Posts: 573
    • View Profile
Re: Quantifying extract & reload
« Reply #2 on: February 28, 2019, 11:49:45 AM »
Hi all,

Wondering if anyone cares to share their insight, tips, and past experience on this.

As some previous posts state, I am looking into extracting data from an AIX system, using TSM/Centera, into a new system using Linux/ICOS. I am trying to put an actual timeframe on how long it is going to take to extract from AIX, and reload into Linux. I have the extract/reload process down mostly, but I am not sure where to start as far as coming up with solid dates.

There is a total of around 7TB of data to extract, totaling 200 Million Documents across 32 application groups. The totals of data in our app groups range from a few hundred MB, to 4.5 TB.

I understand there are SEVERAL variables to consider as far as this entire process goes, but does anyone have any suggestion as to how I can use our logs to actually put some realistic numbers to this.

Thanks in advance.

The only way to know for sure is to take an AG, and look how well it exports, and then how fast it imports.
From my experience, if you have an AG with lots of small documents, with lots of load ids, then it might take an awful long time, compared to an AG with big objects inside.

Since I've nearly never do an "arsdoc get" / "arsload" for such migration, I have very little details to give you.

Yeah, that's what i've been doing. I ran some tests loading / retrieving a few GB of data, and compared that to production.

My concern is the actual reload. Because we are not using TSM in our new environment, and loading a file that's already indexed quicker is obviously quicker than calling ACIF.

As I have mentioned before, I want to stand up a new environment and fix tons of inconsistencies between our objects. If that wasn't the case, I would look into the arsadmin approach that you have mentioned.
#CMOD #DB2 #AFP2PDF #TSM #AIX #RHEL #AWS #AZURE #GCP #EVERYTHING