Majority Leader Steve Scalise is scrambling to lock down the votes to become the next House speaker, but protracted opposition to the Louisiana Republican inside the GOP conference and the numerical realities of the narrowly divided chamber could ultimately derail his bid.

Several senior Republicans see little path to 217 votes, after Scalise won just 113 votes in the GOP conference, which includes three delegates who don’t have a vote on the House floor. Making up that deficit in just a matter of days is an extremely tall order – plus a number of hard-right Republicans say they are dead-set against Scalise, when he can only afford to lose four GOP votes on the floor. At least 12 GOP lawmakers have said publicly they’ll oppose Scalise’s nomination and more have expressed frustration or skepticism about his leadership, more than enough to sink his bid.

House GOP members are gathering again Thursday afternoon behind closed doors, to try to hash out a way forward.

But on Thursday morning, Scalise’s math problem only worsened: At least three more Republicans indicated they would be voting for GOP Rep. Jim Jordan of Ohio in a floor vote for speaker, stiffening the opposition to Scalise.

“I’m not on the whip team, but I can count votes and there’s a pretty big gap right now,” said Rep. Andy Ogles of Tennessee, one of the Republicans who said he would vote for Jordan.

Republicans are worried that Scalise is facing grim prospects of becoming speaker, an impasse that threatens to prolong the GOP’s leadership crisis that has left the House paralyzed and unable to move on any legislation.

Late Wednesday, members of the conference were beginning to weigh how they would handle the potential collapse of his bid, with several GOP sources saying they believe they’d have to consider a new candidate who has yet to run for the speakership.

Scalise spent Wednesday after the vote meeting individually with GOP members as he and his whip operation tried to convince the holdouts to come around, the sources said. He found some success in the outreach, but there’s no indication yet that he can win over enough Republicans to overcome the razor-thin GOP House majority.

Scalise or any other Republican candidate for speaker needs 217 votes to win the speaker’s gavel, a majority of the entire House, which currently has two vacancies. That means Scalise could only afford to lose four Republicans if every member is voting. Democrats are expected to uniformly back House Minority Leader Hakeem Jeffries.

Rep. Dan Crenshaw of Texas, who is expected to back Scalise, sharply criticized members saying that they won’t vote for Scalise.

“I think they should all resign their congressional seat unless they get 100% of their district to vote for them,” he told CNN.

Jordan, who lost the vote for speaker to Scalise on Wednesday, 113-99, said Thursday he wants Republicans to unite around Scalise. “I do and I’ve been clear about that since yesterday,” Jordan said.

But pressed on if he would rule out taking the job if Scalise can’t get there, Jordan didn’t give a clear answer. “I will nominate Steve on the floor and I hope we can unite around a speaker,” the Ohio Republican said.

Former President Donald Trump, who endorsed Jordan, was sour on Scalise’s speakership candidacy in a Fox News Radio interview, pointing to the House majority leader’s recent cancer diagnosis.

“I just don’t know how you can do the job when you have such a serious problem,” Trump said.

The opposition to Scalise inside his party has thrown into doubt how Republicans will get out of their speaker conundrum that’s left them simply unable to govern.

While there was some belief on Capitol Hill that the brutal assault on Israel over the weekend might prompt Republicans to quickly select a leader – House lawmakers were given a classified briefing on Israel Wednesday before the conference vote for speaker – the deep divisions in the conference that led to Kevin McCarthy’s removal last week have now left the quest for a new speaker at a standstill.

Scalise is facing broad skepticism inside the far-right House Freedom Caucus, a key bloc of Republicans who mostly supported the Trump-backed Jordan for speaker, multiple sources told CNN, citing a general lack of trust with Republican leadership. Scalise has been in leadership years, although he is more conservative than McCarthy.

Jordan, who chairs the Judiciary Committee, threw his weight behind Scalise following Wednesday’s vote, saying he was encouraging his supporters to do the same. “We need a speaker and Steve is the guy for that. Like I said, I have offered to give a nominating speech for him,” the Ohio Republican told reporters Wednesday afternoon.

But there was a cohort of lawmakers who expressed staunch opposition to voting for Scalise on the House floor.

“Well, Leader Scalise won, and it’s not over. I’m still throwing my support behind Jim Jordan for speaker. I’m not going to change my vote now or any time soon on the House floor,” said GOP Rep. Max Miller of Ohio.

Scalise’s individual outreach did peel off at least one holdout. Rep. Anna Paulina Luna, who initially said Wednesday that she would vote for Jordan on the floor, met with Scalise and said afterward she felt “comfortable” enough to support his speaker nomination.

While she said he did not make specific commitments, he did assure her that he’ll allow her to “aggressively” do her job on the Oversight Committee, which is part of the impeachment inquiry into President Joe Biden.

But Luna said she would only back Scalise for the speakership on the first ballot. If it went to multiple ballots, she said, “we must find a candidate” the conference can unite behind.

Still, a number of Republicans don’t think that Jordan could be a viable alternative given that he lost to Scalise in the nominating contest, and some Republicans were irritated when he didn’t immediately close ranks behind Scalise.

“If Scalise were not to make it, the next person got less votes,” Rep. Mario Diaz-Balart of Florida said of Jordan. “And by the way, I think, more controversial. So that would not be a good thing for this place.”

Rep. Erin Houchin of Indiana said she doesn’t know if “it will be Jordan or Scalise or even someone else at this point. … I think we’re in uncharted territory, and it’s gonna be very hard to predict.”

Another GOP member said that it would have to be a new candidate altogether, something that would take longer to sort out.

“Steve is nowhere near 217,” said the Republican member.

Leaving the floor without a vote Wednesday, interim Speaker Patrick McHenry tried to be optimistic the House GOP conference would solve the impasse soon.

Asked if there could be a floor vote Thursday, the North Carolina Republican said, “That’s the hope.”

Could anyone get the 217 votes required? He had the same response: “That’s the hope.”

This story and headline have been updated to include additional developments.