PVC is lower cost, and easier to install, however it is also very vulnerable to UV exposure, and damage during installation. PVC is made more flexible through the additions of plasticizers, which tend to leach overtime, and can be incredibly toxic to fish and humans alike.
While HDPE is more expensive, it is also more durable, and fish-safe. However, it is heavy, and not as flexible as its PVC relatives. HDPE includes more cost, but it is possible for one HDPE liner to last nearly 40 years if installed and maintained properly.