It is a large deciduous tree, growing up to 35 m (115 ft) tall (exceptionally to 47 m or 154 ft), with a trunk up to 1 m (3 ft 3 in) diameter. The leaves are 15–30 cm (6–12 in) long, pinnate, with 7–11 leaflets, each leaflet lanceolate, 7–13 cm (2+3⁄4–5 in) long, with the apical leaflets the largest but only slightly so. The flowers are small wind-pollinated catkins, produced in spring. The fruit is a very bitter nut, 2–3 cm (3⁄4–1+1⁄4 in) long with a green four-valved cover which splits off at maturity in the fall, and a hard, bony shell. Another identifying characteristic is its bright sulfur-yellow winter bud.

It is closely related to the pecan, sharing similar leaf shape and being classified in the same section of the genus Carya sect. Apocarya, but unlike the pecan, it does not have edible nuts. It is most readily distinguished from the pecan by the smaller number of leaflets, with many leaves having only 7 leaflets (rarely fewer than 9, and often 11–13, in the pecan). Hybrids with the pecan are known, and named Carya × brownii. A hybrid between the shagbark hickory (C. ovata) is also recognized, and is known as Laney's hickory (Carya ×laneyi).