An infinite family of non-concordant knots having the same Seifert form

By a recent result of Livingston, it is known that if a knot has a prime power branched cyclic cover that is not a homology sphere, then there is an infinite family of non-concordant knots having the same Seifert form as the knot. In this paper, we extend this result to the full extent. We show that if the knot has nontrivial Alexander polynomial, then there exists an infinite family of non-concordant knots having the same Seifert form as the knot. As a corollary, no nontrivial Alexander polynomial determines a unique knot concordance class. We use Cochran-Orr-Teichner's recent result on the knot concordance group and Cheeger-Gromov's von Neumann $\rho$-invariants with their universal bound for a 3-manifold.