arsdoc delete

[rstockton]

When arsdoc is used to delete a document or documents does it remove the date from the segment date range.  Example: If the last date in the segment range is a date greater than current date, and adsoc delete is used to remove the documents that were loaded with that date, does CMOD adjust the segment date range accordingly?

Thanks,
RS

[Alessandro Perucchi]

Hi RStockton,

Well I don't know for CMOD 8.5, but "arsdoc delete" deletes only the index in the library server and nothing more.
Meaning it does not remove the Load ID in TSM, it does not adjust Segment date range.

Best regards,
Alessandro

[rstockton]

Thanks Alessandro,

Document retrieval was running slow and we have found over 500 documents that have loaded with dates ranging from 01/26/2010 to 10/12/2022 which is affecting the searches.  I have looked into the possibility of using arsdoc delete to remove them, but as you stated this only deletes the Index entries therefore leaving the segment date range still incorrect.  I looked at using unload, but it requires the load ID, and it appears that the load ID is assigned for a whole load and not by individual documents therefore it would unload all of the documents for a load ID.  We load several thousands of documents at one time.

RS

[Justin Derrick]

Take this information back to the data source and make your case for the importance of data quality.

-JD.

[Alessandro Perucchi]

Document retrieval was running slow and we have found over 500 documents that have loaded with dates ranging from 01/26/2010 to 10/12/2022 which is affecting the searches.  I have looked into the possibility of using arsdoc delete to remove them, but as you stated this only deletes the Index entries therefore leaving the segment date range still incorrect.  I looked at using unload, but it requires the load ID, and it appears that the load ID is assigned for a whole load and not by individual documents therefore it would unload all of the documents for a load ID.  We load several thousands of documents at one time.

Well there is a workaround, we have thought about it for some customer, but never implemented it yet.
Please be careful if you want to use this idea, and test it, test it, test it, and again test it!!!
And as Justin often says: Backup !!!

So the idea is the following:

Simple version, not so dangerous

- Find the Load ID associated to the document you want to delete
- Look how many documents are in this Load ID
   If only one (this is the document you want to delete) -> unload the LoadID
   If more than one -> arsdoc delete

Now a more dangerous version:

- Find the Load ID associated to the document you want to delete
- Look how many documents are in this Load ID
   If only one (this is the document you want to delete) -> unload the LoadID
   If more than one ->
      - arsdoc delete
      - export all documents from this LoadID (it will export only what is in the Segment) (arsdoc get -cg....)
      - unload the LoadID
      - import the exported data (arsload -X G ....)

And now with this second version you will have the desired behaviour.

But again... if you want to do it. Please test it, test it, test it... and again test it.

I know it's a lot of warning, but if you loose data, then nobody will be happy.

Cheers,
Alessandro

[rstockton]

Thanks Alessandro.

Here is another alternative that I came up with.

1.   Stop Loads
2.   DB2 Backup
3.   Check agid for date ranges
                    a.   Determine if any dates are greater than today?s date
4.   Query Group to verify number of bad documents (arsdoc query)
5.   Delete documents that fall in the date range (arsdoc delete)
6.   Query Group to verify documents were deleted (arsdoc query)
7.   Query for open table (arstblsp)
                   a.  Get the table ID
8.   Close the table (arstblsp)
                   a.   This should readjust the dates in the segment table
                   b.   CMOD will create a new table on the next load for the group
9.   Query to verify table is closed (arstblsp)

Thanks,
RS

[Justin Derrick]

This solves one problem and creates another one -- table segments that are too small, leading to terrible query performance.

Garbage in, Garbage out.  You and your management need to push back and fix the data quality issues you're seeing.

-JD.

[rstockton]

Thanks Justin.

The data problems have been addressed.

RS