Ed has the right idea, breaking up the maintenance into a few steps. You could create a short and simple script to run the steps individually...
echo "Expiring data from caches..."
arsmaint -c
echo "Expiring data from App Groups..."
arsmaint -d
echo "Running table maintenance..."
arsmaint -r
echo "Migrating data from cache to long term storage..."
arsmaint -m
echo "Complete."
Also, try using the -g option to run maintenance on a specific Application Group, and keep an eye on the System Log to see if there are messages relating to the issues it's experiencing. If it fails silently, consider using the arsmaint trace options -1 and -2.
Also, you're running on out-of-support software. Mysterious bugs and bad behavior are to be expected, unfortunately.