We derive and study a Gauss-Newton method for computing asymmetric low-rank product $XX/zz$, where $X /in/R^{n/times k}$ for $k<n$, that is the closest to a given symmetric matrix $A /in/R^{n/times n}$ in Frobenius norm.   When $A=B/zz B$ (or $BB/zz $), this problem essentially reduces to finding a truncated singular value decomposition of $B$.    Our Gauss-Newton method, which has a particularly simple form, shares the same order of iteration-complexity as a gradient method when $k /ll n$, but can be significantly faster on a wide range of problems.  In this paper, we prove global convergence and a $Q$-linear convergence rate for this algorithm, and perform numerical experiments on various test problems, including those from recently active areas of matrix completion and robust principal component analysis.  Numerical results show that the proposed algorithm is capable of providing considerable speed advantages over Krylov subspace methods on suitable application problems. Moreover, the algorithm possesses a higher degree of concurrency than Krylov subspace methods, thus offering better scalability on modern multi/many-core computers.