Selecting the right tool holder type can be a little confusing. Let's review the major tool holder types and their pros and cons.

Collet Chucks

Pros:
Very versatile. By switching collets and you can grab any size shank within the collet's range. Collet chucks are often used to hold drills, taps, end mills, center drills, counter sinks, reamers, and many other tool types. More than any other tool holder type. Tool breakage friendly. Break a tool and worst case it damages the collet. Replace the collet and you're back in business. Decent grip good for medium roughing applications. Good runout, .0003" or better is common. Inexpensive: Collet chucks are fairly inexpensive . Inventory of collets can become pricey, but in the beginning you can purchase only what you need. Many gage lengths available. From short and rigid to very long.

Cons:
Long tool change time. Collet nut and collet should be disassembled every time you put in a new tool for proper cleaning. The inside of the collet can build up with lots of small chips. Balance is decent but with so many moving parts balance can be thrown off. Nose diameter: Depending on the size collet system nose diameters can get fairly large and restrict coolant flow. May also restrict machining next to fixturing or clamps. Grip: Under heavy roughing the shank may slip in collet. This may result in a broken tool and sometimes damaged workpieces.

End mill holders

Pros
Very simple and quick tool changeover. A quality end mill holder balance should be fair to good. Decent to good concentricity. Maritool especially makes end mill holders with tight on-size bores. Shank stays on center even when set screw is clamped. Great grip. Once set screw is clamped on the shank flat, slippage is rare. Cheap. Most tool holder companies will have plenty of stock and fair pricing. Small nose diameters. In most cases smaller than collet chucks. Rigid, good for heavy roughing. Many gage lengths available, from very short and rigid to long.

Cons
Not tool breakage friendly. If a tool breaks and kicks back at the bore on the tool holder it may be damaged. Repair may not be economical. Not versatile. End mill holders can only hold one size shank. If shank is undersized even just a thousandths of an inch the tool will not be on center. The shank must have a flat for set screw to lock on to. If your tool does not have a flat on the shank you will need to grind one on it.

Milling chucks

Pros
Great grip. Tool slippage is rare. Balance is good. Versatile. Bushings (sleeves) are often used to step down the bore so you can hold smaller shank tools. Great concentricity. .0002" or less is common.

Cons
Most milling chucks have a long gage length compared to collet chucks or end mill holders. Pricey: Milling chuck tool holders and sleeves are both pricey. Not very versatile. Even with sleeves the shank must be a common size and at most 6 or 8 sleeve sizes are available. Each sleeve only has one thousandth of an inch range at most. Damage to milling chuck is possible if you clamp on a shank too small. Bore can be crushed and not expand back to normal.

Shrink Fit Tool Holders

Pros
Great grip. Tool slippage is rare. Great concentricity .0001" or less is very common. Small nose diameter. Smallest out of all other types. Excellent for 5 axis work and milling next to clamps or tall part features. No moving parts, best balance. Very well suited for high rpm applications. Medium pricing. Considering the advantages, shrink fit holders are a great value. Good variety of gage lengths and nose profiles. Although shrink fit holders don't get as short as some collet chucks or end mill holders they are available in shorter gage lengths than milling chucks.

Cons
Shrink fit induction heating machine is needed to properly expand a shrink holder. Although shrink holders are decent in price heat shrink induction machines tend to be costly and run on 220 volt or higher. Holders get very hot during the expansion cycle. Great care must be taken to avoid burns. Although changing tools can be quick it may be time consuming waiting for the holder to cool to the touch. That is unless you have an air or water cooling unit. Not tool breakage friendly. If a tool breaks and kicks back at the bore on the tool holder it may be damaged. Repair may not be economical Not versatile. Shrink fit holders can only hold one size shank.

Hydraulic chucks

Pros
Great concentricity, .0001" or better is common. Great for finishing applications. Pressurized fluid inside offers dampening ability which will improve surface finishes of the part and increase tool life. Decent variety of gage lengths equal to shrink fit holders. Fairly slim nose profiles, slightly larger than shrink fit holders. Quickest tool change of all tool holders. Great balance so high rpms are not a problem. Small Allen key releases and clamps new tool.

Cons
For finishing applications only, not for roughing. Holders are expensive. Some holders are known to bleed pressure over time and lose grip, look for manufacturers that vacuum braze components. Not tool breakage friendly. If a tool breaks and kicks back at the bore on the tool holder it may be damaged. Repair may not be economical. Not versatile. Hydraulic holders can only hold one size shank then a sleeve must be used.

Drill Chucks

Pros
Quick tool change. No collets needed. Can hold any size shank within its range. Good for holding drills and taps for light applications.

Cons
Grip is low. Concentricity less than .001" is rare and considered good. Nose diameter tends to be big. Gage length tends to run long. Costly. Cannot hold end mills. Not made to handle side loads. Not tool breakage or slippage friendly. In both cases the jaws will need replacing which is costly and sometimes not economical.