Opening the Black Box: Structural Factor Models with large cross-sections