This page lists the locations of all 24 Infusion phials in the game. Given that there are only 24 of these upgrades, you'll have to pick which stat to prioritize. This depends largely on your playing style.
Infusion 001
In the Fraternal Order of the Raven, head for the chapel and face off with a rather large army of cultists. Once the room is clear, take the Voxophone on one of the pews then check the altar for a Health Infusion phial plus the key to the locked chest in the Lansdowne Residence.
Infusion 002
With the key in hand, you can now bactrack to the Lansdowne Residence which shouldn't be all that far from here. That chest contains another Health Infusion and some Salt.
Infusion 003
Once on the sky-line, you'll want to dismount on the platform where there are enemies shooting at you. Right on the edge of that platform is a telescope. Head inside the building and check the desk in the back room for a Health Infusion phial.
Infusion 004
Past the door behind the giant siphon machine, check out the other two rooms, before heading straight through another set of double doors further down the corridor. Snag the Voxophone from a desk inside, and on the floor to the right behind the desk with the Voxophone is a Health Infusion phial.
Infusion 005
Before you continue past the ticket station, visit the office opposite of where the objective arrow is telling you to go. You'll find a Voxophone on the desk inside. Next, enter the ticket station from a passage in the hallway to retrieve a Health Infusion phial from somewhere in the room.
Infusion 006
In the Welcome Center before sending Booker to lift the metal gate, turn left to find a locked door that can be picked open. The Infusion is inside this office.
Infusion 007
In the secret room of The Fellow Traveler bar. Use the Code Book from the cannon in the Tickets building to decipher the message in bar's rest room then pull the derby hat in the adjacent toilet stall to open the passage.
Infusion 008
For starters, go around behind the counter and enter the ticket booth (the one with a corpse) to score a Voxophone from the table in the back. Head to the next section where there is a large statue of Comstock. Wait for Slate's men to come out of a side room and kill them. Go in that room (there will be a smashed statue of Slate here) to secure a Health Infusion phial.
Infusion 009
After the battle in the Courtyard, face the Slate's message on the wall and turn left toward an open door. In that workshop Voxophone along with a Health Infusion phial inside a green tool box.
Infusion 010
After activating the gondola with the Shock Jockey, defeat the enemies and ride it to the Aerodrome. There is an Infusion phial on the desk inside the Zeppelin.
Infusion 011
Pursue Elizabeth through the harmless dimensional tears, and once she gets nabbed, check the desks behind the partition on the left side to find a Voxophone. Follow the objective arrow to the sky-line. Explore the port, kill enemies and leave Elizabeth to rot in her cell for a while longer. Look for is a pair of large, red double doors that leads to a cell block with an Infusion phial.
Infusion 012
In the upper floor of the building, there is a locked Employees Only office that requires five lock picks to open. You should have plenty of these by now provided that you ignore any safe and religiously loot every new area you're in. On the desk inside that office is an Infusion phial.
Infusion 013
The office with a fireplace at the upper floor of the club has a Gear in it, while on the lower floor is a private bar located next to a Dollar Bill vending machine. This one has an Infusion phial inside.
Infusion 014
In the alternate Plaza of Zeal, head to the building with the coded message. Set the big round clock to the right, opening a safe with an Infusion and a Voxophone. The Code Book for this is in the Good Time Club's disposal facility before you open the tear.
Infusion 015
Make your way down the street until you're alerted a robbery in progress. Kill the two ruffians stealing from the homeless men and check the crate behind them for an Infusion phial.
Infusion 016
Moving forward to the second floor just past the medical kits, turn into one of the corners and you'll find a door that requires 5 lock picks to open. Inside this room is an Infusion phial and a Gear.
Infusion 017
There is another room past the medical kits on the second floor that contains a locked chest; the key for it is found in the basement of the Graveyard Shift bar back in Shanty Town. The chest houses another Infusion phial.
Infusion 018
After the story event with Elizabeth and Daisy Fitzroy, turn to Booker's left for an Infusion phial.
Infusion 019
Just past the Hairpin-locked door leading to Port Prosperity is another door that can be picked open (requires 3 lock picks). Go in there and pick up the Infusion phial from the washtub at the end of the corridor.
Infusion 020
After getting the Voxophone in the Founders Books basement, go back to the Salty Oyster bar. Press the button under the register to open the secret room. In it is a wide variety of valuable loot including an Infusion phial and the Return to Sender Vigor on the counter and a Voxophone in the washtub.
Infusion 021
Locate the Wilson Bros. & Company Bottling Works next and break in with 3 lock picks. There is an Infusion phial by the window inside.
Infusion 022
In the Lutece Lab when seeking out the first of three tears, visit the bed room upstairs next for an Infusion phial plus another Voxophone sitting on the floor between the bed and the night stand.
Infusion 023
In the Bank of the Prophet while seeking out the second of three tears, take a left at the vault and enter the other door in the next room. There, the Code Book comes into play in the section that says "HOARDER" on the wall. Use the typewriter to open a secret room full of valuable loot including this Infusion.
Infusion 024
After Booker regroups with Elizabeth, go back to that door that previously said "Elizabeth Busy. Can't Lockpick." and open it. Inside the doctor's office is the last (24th) Infusion phial and the 76th Voxophone.