The man is accused of sexually assaulting a woman on the same day he was released under the government’s early release programme.
He is said to have relapsed in Sittingbourne, Kent, and was subsequently arrested at an address in south London.
He was one of more than 1,700 inmates released from prison last week to ease prison overcrowding. The government says terrorists and convicted sex offenders are not held in prison.
The former inmate appeared in local court on Thursday charged with sexual assault. He is due to appear in court next month.
He has been ordered back to prison.
A Ministry of Justice spokesman said: “The government took over a prison system in crisis and on the brink of collapse. If that had happened, the courts would not have been able to hold trials and the police would not have been able to make arrests.
“We had no choice but to introduce emergency measures, release some prisoners weeks or months early, rule out many offences and impose tough conditions of detention.
“We cannot comment on the details of individual cases, but anyone who breaches the conditions or commits further offences will be punished.”
The government’s policy to ease prison overcrowding means some prisoners have to spend time behind bars. It has been temporarily reduced from 50% to 40%.
Around 1,750 prisoners were released from prisons in England and Wales on Tuesday.
Pictures show inmates being showered with champagne by friends at the prison gates as they are released.
Despite government efforts to prevent domestic abusers from being released early from prison, the policy only excludes certain crimes, such as stalking, so the restrictions do not apply to all offenders with a history of such violence.
Perpetrators convicted of other crimes, such as threats to kill or actual bodily harm, may also be eligible for early release.