The only difference between installing a Library and Object server is the database component. So, to install a CMOD Object Server, simply skip the steps involving the (DB2/Oracle/MS-SQL) database. In terms of configuration, you'll need to change ars.cfg to configure the server as ab Object Server.
Having said that... There are very, very few situations in which managing separate object and library servers makes sense anymore. Initially, the rationale for separate library and object servers was to distribute the work of loading files back when CPUs ran around 100Mhz, and to reduce cost and latency when network bandwidth between branch offices was both slow AND prohibitively expensive. Now, those issues only exist for the very largest of systems (with petabytes of data and hundreds of thousands of users).
Is there a specific reason why you want to go with separate Object and Library servers?
-JD.