OnDemand User Group

Support Forums => OD/WEK & JAVA API => Topic started by: ODCoder on March 12, 2019, 02:21:12 AM

Title: ODWek Java API - recreateHit issues <Resolved>
Post by: ODCoder on March 12, 2019, 02:21:12 AM
Hello all,

I have successfully managed to return a list of documents based on a search criteria.  The next step is the user selects which document they want to return and it displays the document.    So I have returned a list of documents, which includes the (I assume encrypted) document ID ODHit.getDocId().  Now I'm trying to get the document using recreateHit as follows:

Code: [Select]

try
{
odHit = odFolder.recreateHit(inDocID);
}


I was expecting to maybe have an issue because im sending back an encrypted docid, but to my surprise, I get the follow error in the exception:

Code: [Select]

Error 500: java.lang.RuntimeException: org.apache.cxf.interceptor.Fault: The user specified does not have access to any folders


Is that just a masked way of saying that its encrypted?  Do I have to do a bit more than just recreateHit?  I'm not seeing much in the documentation that suggests I need to do more.

Any idea's?

Thanks very much!

DB
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 05:49:36 AM
What version of OnDeamand are you running?

My guess is you need to switch to getOpenDocId() as the docID's are now tied to a user. 

Thanks,

RR



Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 06:16:54 AM
Ohhh, ok that kind of makes sense.  OnDemand 10.1.  I will give that a try now. Thank you!
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 06:27:29 AM
rjrussel, I've been looking at this call.  It doesn't accept any parameters, it just simply returns a string with the new DocID, so what do you pass it?  I can't pass it an ODHit with the user tied docid, or I just get permission denied as before?

Thanks again for your help.

DB
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 07:07:47 AM
See if this (albeit crude) example helps:

     odServer.logon("shortrib", "admin", "password");
   
   ODFolder folder = odServer.openFolder("Baxter Bay");
   Vector<ODHit> hits = folder.search();
      
   ODHit odHit = (ODHit)hits.get(0);
         
   String hit = odHit.getOpenDocId();
   odServer.logoff();
   odServer.terminate();
   odServer = new ODServer(odConfig);
   odServer.initialize("TestOpenDocId");
   odServer.setPort(0);
   odServer.logon("shortrib", "rjrussel", "password");
   folder = odServer.openFolder("Baxter Bay");
   ODHit hit2 = folder.recreateHit(hit);
        System.out.println("Recreated:" + hit2.getDocumentName());


Keep in mind in order for recreate hit to work the user recreating the hit MUST have permissions to the document. If that user doesn't you will still get an error.

Thanks,
RR
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 07:16:12 AM
Right, I'm with you now, I didn't think about it that way around.

Thanks RR that's perfect!
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 07:30:34 AM
Hi RR, that didn't work.

Here is the situation.  I'm wrapping the java API into rest calls.  The first rest call uses search(), and returns JSON with a list of doc ID's and metadata.  The 2nd REST API, uses the DOC ID to retrieve the document, as base64 JSON.

So these are 2 separate connections, potentially minutes apart, maybe that's the issue here?
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 08:13:59 AM
So you confirmed you are using getOpenDocId()?

What error are you getting when you attempt to recreate the hit?
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 08:49:42 AM
Correct, I'm using getOpenDocID.

The error I'm getting is-

Code: [Select]


Error 500: java.lang.RuntimeException: org.apache.cxf.interceptor.Fault: The user specified does not have access to any folders


Obvoiusly teh 500 apache stuff is just hte webservice error.  The "The user specified does not have access to any folders" is the OD error.

For reference, here is the first part of hte call:

Code: [Select]

ODServer odServer;
ODFolder odFolder = null;
ODHit odHit;
ODHit odHitNew;
ODConfig odConfig;
byte[] data_from_server;
File file;

odConfig = new ODConfig();
odServer = new ODServer(odConfig);
odServer.setPort(inPort);
odServer.initialize( "TcSQLSearch.java" );

odFolder = odServer.openFolder(inFolder);

try
{

odServer.logon(inServerName, inUserName, inPassword);
}
catch (Exception ex)
{
throw new WebApplicationException(Response.ok(ex).build());

}

try
{
odHit = odFolder.recreateHit(inDocID);


}
catch (Exception ex)
{
throw new WebApplicationException(Response.ok(ex).build());

}

Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 09:39:10 AM
I see it now....

You should really log on before you try and open the folder  ;D

             odFolder = odServer.openFolder(inFolder);
      
      try
      {
         
         odServer.logon(inServerName, inUserName, inPassword);
      }


Swap those two calls and watch the magic happen.

-RR
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 12, 2019, 09:48:59 AM
Yes your quite right! And I spotted that too, probably around the time you did :)

Now I'm getting:

{"errorMsg":"Unable to restore hit from DocID","errorId":9,"message":"Unable to restore hit from DocID","localizedMessage":"Unable to restore hit from DocID","stackTrace":[{"className":"com.ibm.edms.od.ODFolder","fileName":"ODFolder.java","lineNumber":2180,"methodName":"recreateHit","nativeMethod":false},

Regardless of which method I use to retrieve the docid
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 10:04:53 AM
Assuming you have permission to the doc it's probably an encoding issue.....

Create yourself a little stand alone app. Get it working and then incorporate into your rest service.

In most cases the docID you include in your rest response should be URL encoded. Something like:
URLEncoder.encode(odHit.getOpenDocId(), java.nio.charset.StandardCharsets.UTF_8.toString()))

However, when you pass it back in to retrieve the content you shouldn't have to decode....

You're very close.... Good luck

-RR

Title: Re: ODWek Java API - recreateHit issues
Post by: Alessandro Perucchi on March 12, 2019, 01:20:11 PM
Well if you get the docid with a user, then use another user to recreate the docid, then it will never work in a default setup.
In order to make it work, you must use an encryption key when you get the docid, and then when you use the other user, you use the same key to decrypt the docid you want to recreate.

Look at this method, because this is exactly what you want to use : ODServer.setEncryptionKey(...). Here is what the Javadoc says:

Quote
Document identifiers (docids) are now encrypted. This method can be called after a user logon to specify the encryption key to be used for docid generation. If this method is not called ODWEK will generate the encryption key. In both instances, the encrypted docid is bound to the userid currently logged in when the docid is generated. This ensures that only this user can retrieve the contents of the document using the docid. This userid binding to the docid can be circumvented by calling this method prior to user logon. When used in this fashion it is imperative that the calling application ensure the docid is only used to retrieve the contents of the document by users with the proper permissions.

Title: Re: ODWek Java API - recreateHit issues
Post by: Alessandro Perucchi on March 12, 2019, 01:22:54 PM
Hmmmm I'm reading the new method ODHit.getOpenDocID(...), interesting, so it means the encryption key method is not anymore needed! Thank you! I will go to bed less stupid than this morning! :-D
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 12, 2019, 05:16:24 PM
Using the getOpenDocID() comes with some overhead so it should be used wisely. Since there is no user tied to it we must makes sure the user who now has possession of the docid is authorized to see the content they are requesting.

So in the case of REST based services one should be using connection pooling (whenever possible) and therefor the getOpenDocID really shouldn't be needed..... At least in most cases.....
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 13, 2019, 01:12:06 AM
Thanks all for your replies.  I encoded and decided which worked fine, however I’m still getting a permisissions issue.

I will try an app to do the same thing to see if it’s just a web services issue.
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 13, 2019, 08:22:39 AM
I've done some work on this and written a simple java application running on the server to replicate the REST API code.  After running it against the same docid, using odHit.getOpenDocId(), the following error occurred:

Code: [Select]

Logging on to localhost...
com.ibm.edms.od.ODException: Unable to restore hit from DocID
        at com.ibm.edms.od.ODFolder.recreateHit(ODFolder.java:2180)
        at com.test.logon.main(logon.java:45)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
        at java.lang.reflect.Method.invoke(Unknown Source)
        at org.eclipse.jdt.internal.jarinjarloader.JarRsrcLoader.main(JarRsrcLoa
der.java:58)


Pretty much the same error as I'm getting in the web service.  So to summarize:

Using getOpenDocId -or- getDocId against the Hit object, to retrieve a document ID, then, use the same docid on recreateHit, appears to not work on its own, so I'm guessing I'm either doing something wrong, or I'm missing something?

Any idea's?
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 13, 2019, 08:41:05 AM
I would turn on ODWEK trace, rerun your test and then look for more detailed errors there. Report back..... Also, what is the exact version you are running, 10.1.0.?
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 13, 2019, 10:00:22 AM
Trace as follows:

Code: [Select]
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(7210)Java_com_ibm_edms_od_ArsWWWInterface_apiRecreateHit:Enter session id=5681056
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(3234)apiP_OpenFolder:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1895)apiP_GetStringUTFChars:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1923)apiP_GetStringUTFChars:Return rc=0
7392:8532 03/13/2019 16:58:38:158162 INFO ars3wapi.cpp(3276)apiP_OpenFolder:Current state Opening Folder=Statements
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvfl.c(511)CsvOpenFolder:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvlg.c(2986)CsvFolderQuery:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvlg.c(3016)CsvFolderQuery:Return csv_rc=0,CSV_RC_OKAY csv_msgid=0,CSV_MSG_NO_MESSAGE
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvfl.c(581)CsvOpenFolder:Return csv_rc=0,CSV_RC_OKAY csv_msgid=0,CSV_MSG_NO_MESSAGE
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(3303)apiP_OpenFolder:Return csv_rc=0,CSV_RC_OKAY csv_msgid=0,CSV_MSG_NO_MESSAGE
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1895)apiP_GetStringUTFChars:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1923)apiP_GetStringUTFChars:Return rc=0
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1941)apiP_RestoreHitFromDocId:Enter
7392:8532 03/13/2019 16:58:38:158162 INFO ars3wapi.cpp(1970)apiP_RestoreHitFromDocId:Current state byte_buffer=O1qom06pSqO5gv1w%2F9T8gU9ffb2EDfSa9cuN8jAO4kHbfueaJ8vOUPzO9C0gQoeuhBNm0bF1ol4kSBqlXJUwsrcoZ0R4S8Ng0fwhAQlgrq6TqOqQKROFxk57gJVLUjvuFWqJlldEIKLDrv9vK1D1OjCP7WtIdsTummIaBw4ZDx%2BPba25Z7b%2FyWWkN9hJ3Q8r4CJlqs8kwIR432IVwxGeLinObmDjbX%2FoIcs2kj%2BOuIEGgXh0AYztI9ayHMjhu2IqXu
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(2766)CsvRestoreHitFromDocId:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(1956)CsvMaxDocIdLength:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(1985)CsvMaxDocIdLength:Return len=6046
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(2164)CsvP_ICCDecrypt:Enter
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(2207)CsvP_ICCDecrypt:Return arccs return code=6,ARCCS_FAILED
7392:8532 03/13/2019 16:58:38:158162 FLOW arscsvht.c(2982)CsvRestoreHitFromDocId:Return pCsvHit=0000000000000000
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(1997)apiP_RestoreHitFromDocId:Return
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(713)apiP_setReturnCodeAndMessage:Enter
7392:8532 03/13/2019 16:58:38:158162 ERROR ars3wapi.cpp(7423)Java_com_ibm_edms_od_ArsWWWInterface_apiRecreateHit:Current state rtn.RC=9 extId=0 pMsg=Unable to restore hit from DocID
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(789)apiP_setReturnCodeAndMessage:Return
7392:8532 03/13/2019 16:58:38:158162 FLOW ars3wapi.cpp(7436)Java_com_ibm_edms_od_ArsWWWInterface_apiRecreateHit:Return session id=5681056 (rc)=1

Possibly a decrypt issue?
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 13, 2019, 10:10:48 AM
Yes, that docID looks urlencoded.
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 13, 2019, 10:18:34 AM
Yes your right, the rest api was returning url encoded, if I look at the raw json and put that in, it seems to work fine.  So I seemed to have proven the code works, just not in a rest api.
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 14, 2019, 01:44:41 AM
I've spent some more time on this.

Since its the same user that's searching and retrieving the document, both types of getDocID work from the java app, neither work from the rest app.

I have also tried URLEncoding and decoding, but that made no difference either.

I can't seem to find a way to turn tracing on with a rest api.  Even turning up tracing on WAS didn't seem to help.  So I have no other clear indication of what is going on, other than:

Code: [Select]
"errorMsg": "Unable to restore hit from DocID",
    "errorId": 9,
    "message": "Unable to restore hit from DocID",
    "localizedMessage": "Unable to restore hit from DocID",

Any idea's would be great.

Thank you!

DB
Title: Re: ODWek Java API - recreateHit issues
Post by: rjrussel on March 14, 2019, 07:54:59 AM
1. If its the same user (connection pool) make sure you use getDocID.
2. The docID should be returned in the response URLEncoded
URLEncoder.encode(odHit.getDocId(), java.nio.charset.StandardCharsets.UTF_8.toString()

3. Whether you need to decode the docID before you call recreateHit depends on how you send the data back to your rest api.

It shouldn't be to hard to do some logging and see how it gets passed in so you can determine whether you need to decode or not.
Title: Re: ODWek Java API - recreateHit issues
Post by: ODCoder on March 15, 2019, 12:31:58 AM
I did all of that, used both GetDoc commands to be sure, encode and decode.

One thing I have noticed is that it appears to add a + where there is a space, before and after encoding.  But even if I strip that out it doesn’t seem to make any difference.

Will keep playing, this shouldn’t be that hard!
Title: Re: ODWek Java API - recreateHit issues <Resolved>
Post by: ODCoder on March 21, 2019, 02:44:00 AM
Hi all,

Thanks for everyone's help, especially RR!

I eventually got there through trial and error.  I'm still unsure exactly why, but URL encoding the original DOCID in the GET that searches the folder, then on the GET that retrieves the document, do not URL decode, and replace any <space> with a + sign, seemed to work.  Again, I'm just not familiar enough with HTTP encoding to understand a straight forward decode didn't work.  Also the removal of the + and the insert of the space is odd.


So a combination of the string manipulation and the getOpenDocId(), appears to be the final solution.

Regards


DB