Officials say it was probably caused by a soft raw milk cheese called Ouleout from Vulto Creamery in New York state.
The cheese was stocked by a Whole Foods shop in Fairfield, Connecticut, and may also have been available in specialised cheese shops.
The creamery recalled several soft cheeses on Tuesday.
Six cases of listeria have been recorded in Connecticut and Vermont, where the deaths occurred, as well as New York and Florida.
Vulto Creamery has not made any comment.
The company says it makes small batches of handmade cheese using raw milk from local dairy farmers. Its founder started out by making cheese in his apartment in Brooklyn.
Vulto describes the Ouleout cheese as a “semi-soft washed rind cheese” made from unpasteurised milk that tastes “pungent and meaty”.