JERUSALEM—Israel’s new finance minister, the leader of the country’s far-right Religious Zionism party and an advocate of outright annexation of at least parts of the West Bank, said he plans to spend billions of dollars building infrastructure and investing in Jewish settlements in the territory.

“We will lead the development, formalization and strengthening of our grip in these areas of our homeland, Judea and Samaria,” said Bezalel Smotrich as the new government was sworn in last week, using the biblical name for the West Bank.