Activist investor William Ackman's firm Pershing Square Capital Management has acquired a 9.8 percent position in $22 billion industrial gas company Air Products & Chemicals, CNBC reported on "Squawk Box" Wednesday morning.

The stake is valued at $2.2 billion, according to Ackman, and is his largest investment ever by cost. The 13D filing is expected Wednesday.

It comes at a time when Ackman finds himself underwater on last year's short bet against Herbalife stock. In addition to the Air Products acquisition, he also told CNBC that he hasn't "covered a single share" of Herbalife.

In April, Ackman -- a J.C. Penney board member -- acknowledged the shortcomings of ousted Ron Johnson, the retailer's chief executive he had handpicked to turnaround the company. The stock is down nearly 18 percent this year.

Back in July 2012 at the CNBC-Institutional Investor Delivering Alpha Conference, Ackman talked about the big stake he took in Procter & Gamble and then pushed for management changes. P&G has since brought A.G. Lafley out of retirementto lead the company. Its stock has risen 23 percent since Ackman's involvement.

As for Air Products, "it's a great business that is undervalued," Ackman told CNBC Tuesday -- citing a very diverse customer and product base, high barriers to entry, and substantial pricing power. "We have some ideas on how to add value."

Some of the diverse businesses of Air Products include supplying industrial gases, performance materials, equipment and technology used in industries from steel production to food processing to electronics.

Related: The 10 Fastest-Growing Industries for Small Business

Similar to the railroad industry, where Ackman has had much success in his activist campaign against Canadian Pacific, Air Products has few real competitors, including France's Air Liquide, Germany's Linde, and U.S. company Praxair.

Ackman said he has ideas on how to make Air Products more valuable. He wouldn't elaborate on those plans, but a look at his involvement in the Canadian Pacific may be a playbook for what he plans to push with Air Products, which Ackman called a "[Warren] Buffett-like business."

At Canadian Pacific, Ackman bought a 14 percent stake in 2011, then proposed bringing in a new management team. The board resisted. Ackman brought a proxy contest, winning an overwhelming majority of the vote and garnering a majority of the board seats. He brought in a new CEO, experienced rail executive Hunter Harrison, last June. Canadian Pacific shares have more than doubled since Ackman bought in.

Earlier this month, Ackman told investors in a letter that he had built up a major position in a large-cap, investment-grade U.S. corporation. Since then, some had speculated Air Products might be that company, and its stock has risen from the low $90s to close above $105 Tuesday.

Just last week, Air Products adopted a poison-pill plan, citing unusually high volumes of stock trading. The plan gives existing shareholders the right to accumulate deeply discounted new shares should another group acquire 10 percent or more of the shares outstanding without board approval.

The poison pill prevented Ackman from taking a larger stake in Air Products. "I was a little too cute in my letter to investors. I didn't expect people to be running around looking for the company. I think it alerted the market," said Ackman, acknowledging the speculation had complicated his plans. "It also made the stock more liquid so it enabled us to buy about 10 percent of the company, but we would have bought more if they hadn't put in the poison pill."

Related: Secret's Out: Now You Can Tell the Whole World You're Raising Money for Your Business


More from CNBC