Yes, but this all depends on your router. If your router support PPTP or OpenVPN, it should be quite possible if you follow the instructions for your specific router.

One method is to use a custom firmware such as dd-wrt. However, you should be warned that doing so can break your router if not done properly. Make sure your router is compatible and that you have the correct version of the firmware.

https://wiki.btguard.com/index.php/PPTP_dd-wrt

https://wiki.btguard.com/index.php/OpenVPN_dd-wrt