We investigate the genetic basis of panicle architecture in switchgrass in two mapping populations across a latitudinal gradient, and find many stable, repeatable genetic effects and limited genetic interactions with the environment. Grass species exhibit large diversity in panicle architecture influenced by genes, the environment, and their interaction. The genetic study of panicle architecture in perennial grasses is limited. In this study, we evaluate the genetic basis of panicle architecture including panicle length, primary branching number, and secondary branching number in an outcrossed switchgrass QTL population grown across ten field sites in the central USA through multi-environment mixed QTL analysis. We also evaluate genetic effects in a diversity panel of switchgrass grown at three of the ten field sites using genome-wide association (GWAS) and multivariate adaptive shrinkage. Furthermore, we search for candidate genes underlying panicle traits in both of these independent mapping populations. Overall, 18 QTL were detected in the QTL mapping population for the three panicle traits, and 146 unlinked genomic regions in the diversity panel affected one or more panicle trait. Twelve of the QTL exhibited consistent effects (i.e., no QTL by environment interactions or no QTL × E), and most (four of six) of the effects with QTL × E exhibited site-specific effects. Most (59.3%) significant partially linked diversity panel SNPs had significant effects in all panicle traits and all field sites and showed pervasive pleiotropy and limited environment interactions. Panicle QTL co-localized with significant SNPs found using GWAS, providing additional power to distinguish between true and false associations in the diversity panel.