The holes you drill are often just a tad too big or too small for dowels and fasteners.
Buy an indexed set of #1-#80 numbered or A-Z lettered drill bits. A common part of a machinist’s tool chest, these twist bit sets include many bits that fall between standard fractional sizes. For example, say you want to drill a hole for a hinge pin on a jewelry box. A standard 1/8" bit makes a hole that’s too tight for the pin, but a 9/64" hole’s just too loose. Both the #29 and #30 size bits in a numbered bit set fall between 1/8" and 9/64", allowing you to drill a hole that’s just right for your hinge pin.