I wouldn't be too worried if you do not have any cuts in your skin while you were in physical contact with your hands.
1) Human Immunodeficiency Virus - HIV can only survive in bodily fluids. It does not survive very long in open air. Unless you had contact with the vaginal or seminal fluid from the genitalia of the affected individual, it's unlikely were exposed to the virus in the first place.
2) Your skin has strong natural barrier against viruses and bacterial and HIV (virus) cannot penetrate through the layers of your skin to reach the vulnerable tissue which HIV thrives on.
So, unless you have a cut on your hands/fingers while you were in contact with the bodily fluid of the affected individual, your risk is almost negligible from just touching the genitalia of an affected individual.
But however, because HIV patients can have co infections with viruses like Human Papilloma Virus (HPV) or Human Herpes virus (HHV), and these viruses can penetrate through the skin to cause skin infection, some times you can come down with warts and blisters respectively after contact of affected areas.
Do monitor for rashes and seek help if you are worried.
