You can shop for deals on VPSes at lowendbox.com. But if you're trying to run a business, this is a waste of time. Find a provider which is highly reliable, which can also automatically rebuild all failed infrastructure with no intervention needed. That eliminates 99.999% of the providers out there. You're paying a premium to never have to think about your tech again, so you can focus on the business.
Besides using the free tier and other AWS services which are practically free at low uses, you can use cost effective options like Fargate Spot Instances and EC2 Reserved Instances. I highly recommend Fargate over running instances. Use Lambda with CloudWatch triggers if you need to schedule occasional jobs (or use Fargate's feature for that). Try to avoid heavy reliance on caches, ElastiCache is kind of a rip off. Move as much content to static as possible, use CloudFlare to reduce bandwidth costs. If you're gonna serve over S3, you might as well front it with CloudFront as it's actually cheaper due to caching at the edge, and also more reliable. ALBs are expensive but very useful for APIs as well as autoscaling (if you have to run instances, run them with an ASG, which also means having versioned AMIs)
Besides using the free tier and other AWS services which are practically free at low uses, you can use cost effective options like Fargate Spot Instances and EC2 Reserved Instances. I highly recommend Fargate over running instances. Use Lambda with CloudWatch triggers if you need to schedule occasional jobs (or use Fargate's feature for that). Try to avoid heavy reliance on caches, ElastiCache is kind of a rip off. Move as much content to static as possible, use CloudFlare to reduce bandwidth costs. If you're gonna serve over S3, you might as well front it with CloudFront as it's actually cheaper due to caching at the edge, and also more reliable. ALBs are expensive but very useful for APIs as well as autoscaling (if you have to run instances, run them with an ASG, which also means having versioned AMIs)