All-solid-state batteries (ASSBs) based on solid polymer electrolytes are among the most promising candidates for next-generation batteries because of their improved safety and energy storage performance. However, the low ionic conductivity of polymer electrolytes at ambient temperature is their main limitation. Composite polymer electrolytes constituted of an organic polymer matrix and an inorganic filler material can mitigate limitations without compromising the advantages of the respective materials by improving the interfacial contact with electrodes and suppressing the lithium dendrite growth. In this work, two different types of fillers, passive and active fillers, were compared to investigate the effect on the ionic conductivity of PEO-based composite electrolytes. Both passive and active oxide filler particles, ZrO2 and Li7La3Zr2O12, were functionalized with silane-based ligands to stabilize the fillers to obtain a homogeneous distribution in the composite. The distribution of filler in the composite electrolyte was observed by scanning electron microscope. The effects of filler type and filler content within the composite electrolyte on its ionic conductivity were studied through impedance measurement at various temperatures. For example, a solid polymer electrolyte with 15 vol. % of ZrO2 achieved a high ionic conductivity of 6.65 × 10-4 S cm-1 at room temperature, which is a sufficient value to operate all-solid-state cells. Depending on the filler type, the optimum content to enhance the ionic conductivity was elucidated. Furthermore, the mechanical and thermal stability of the polymer electrolyte showed to be substantially improved by the addition of the fillers. This study suggests that passive fillers can serve just as well as active fillers for enhancing the ionic conductivity, and the effect of prevention of crystallization has a stronger influence on the conductivity than the ion conduction of the filler.