If the determinant is exactly zero, no solution can be computed. If the determinant is very small, like 0.0001 or 0.00001, the results may be unstable, in the sense that a small change in some variable values may produce large changes in results. This is so for any regression or correlation problem, including factor analysis.
This is due to some variable/s being an almost exact linear function of one or more other variables (for instance, a total score in a scale is usually an exact linear function of the items in the scale; but suppose the values of the items originally had decimals that were rounded, then the total score (obtained with the unrounded values) may be ALMOST an exact linear function of the rounded items.
You may try eliminating one variable or another (choose whichever ones you deem less important, or more closely related, conceptually, to other variables), and see whether the value of the determinant significantly increases. Otherwise, you may use your data as they are, with det=0.0001, but beware of the instabilities. These instabilities increase as samples get smaller, and
are very large with relatively small samples (i.e. less than, say, 50 cases per variable involved in the factor analysis).