The deputy commons speaker and Tory MP says he regarded the men who accused him of rape and sexual assault as friends.
The 55-year-old Conservative MP for the Ribble Valley was released on Saturday night after being arrested at his home in Pendleton, Lancashire. He was given police bail until June 19.
Lancashire Police said the offences were alleged to have been committed in Pendleton between July 2009 and March 2013. Both alleged victims are in their 20s.
Police were seen carrying out searchers on Mr Evans’s home and on a car parked outside.
In a short statement outside his home on Sunday morning, Mr Evans said: “Yesterday, I was interviewed by the police concerning two complaints, one of which dates back four years, made by two people who are well known to each other and until yesterday, I regarded as friends.
“The complaints are completely false and I cannot understand why they have been made.
The complaints are completely false and I cannot understand why they have been made. Nigel Evans
“I appreciate the way the police have handled this in such a sensitive manner and I would like to thank my colleagues, friends and members of the public who have expressed their support and, like me, a sense of incredulity at these events. Thank you very much.”
Mr Evans spoke briefly to reporters later on Sunday afternoon before going for his “Sunday lunch pint” in the pub next door to his house.
He said that apart from the “very recent loss” of his brother it had been “the worst 24 hours of my life”, and thanked his friends and supporters for their “amazing and overwhelming” support”.
Mr Evans’s solicitor Adrian Yalland said he did not intend to stand down as deputy speaker or as an MP.
But it was reported that speaker John Bercow had agreed to a request from Mr Evans that he should be excused his duties chairing the Queen’s Speech debate on Wednesday due to the “inevitable added time pressure” that had been placed upon him.
Defence Secretary Philip Hammond said it could be “very difficult” for Mr Evans to carry on as deputy speaker while the matter was being investigated.
Foreign Secretary William Hague called Mr Evans “a popular and well-respected member of Parliament and deputy speaker”.
Mr Evans has been MP for the Lancashire constituency since 1992 and was elected as one of the three Commons deputy speakers in June 2010.
Later that year, he revealed he was gay, saying that he was “tired of living a lie”.
Mr Evans claimed a Labour MP had threatened to “out” him and that he had decided to be open about his sexuality to help others in a similar position.