If an adversary has already gained the initial access to your infrastructure, his ability to spread his reach to more resources (lateral movement), will be largely depend on network visibility, therefore, to reduce the impact area (also known as Blast Radius) it is vital to separate the networks using Virtual Private Clouds (VPCs), exposing in public subnets only the resources that need public access (such as a web server for a public application), and leaving other resources, such as databases in private networks.
If an instance on the private subnets needs to connect to Internet to update, you can configure a NAT Gateway that will only allow outgoing access, without moving the instance to a public subnet and providing a Public IP address.
You should connect a VPC to an on-prem datacenter using VPN (ideally over DirectConnect) ONLY IF the vpc requires access to on prem resources. It’s not advisable to connect all the VPCs for future usage.
As an example, in a web application that uses an application load balancer, the databases (in this example Amazon Aurora multi-master) should stay on a private subnet and Security groups should specify specifically that only the Web Server’s SG can connect to the Databases:
Isolating the workloads in multiple accounts and multiple VPCs you can reduce the potential impact of an attack containing it to a single VPC/Account.
Is ok to establish VPC Peering when needed, but for security reasons it’s convenient to limit the usage of vpc peering when it’s not required.
Note: Keep in mind that for environment types (Dev/Test/Prod) it’s advisable to split in multiple accounts to avoid competetion for the resources in the account causing potential hits to the service limits that are “per-account”