I would recommend removing it safely and expediently. You may try to irrigate the fragment out with normal saline, or tap water if that is handy.
If you still fail to remove the fragment, then consider a review by your attending Ophthalmologist who will assist with this. Very frequently, patients continue to experience a persistent foreign body sensation due to trauma to the ocular surface that is induced whilst trying to remove the "fragment".
A review by your Ophthalmologist will help to confirm this, and if the fragment is no longer present, he will be able to reassure you of that fact.
