Column headers are the letters in gray cells (or in cells of another color, depending on your Microsoft Windows display settings) across the top of the spreadsheet, and row headers are the numbers in gray cells down the side of the spreadsheet.
Drag the boundary until the row or selection is the height you want, or, to make the row height fit the contents, double-click the bottom row boundary.