The removal of an ICL lens implant is usually easier and faster than the ICL lens implantation procedure. The serious risk involved is of intra-ocular infection which can occur in 1 in 1,000 cases.
But yes, you will be able to wear your contact lenses after around 2 - 3 months after the ICL lens removal. In the meantime, you can wear your glasses.
Hope that helps!