Picture the following example, you have your VPC and you have a business partner with their own VPC. You successfully have a VPC peering configured, bidirectional communication, and life is good:
Now what if behind their VPC they had an on-premise network not hosted with AWS that you need to reach as well? Simple right, just have your VPC traffic route to their VPC, and have their VPC route your traffic to their on-premise network via a few static routes:
This is where you'll run into an issue, by default Amazon does not allow traffic not originated from within a VPC to be routed out of its own network. Essentially you're not allowed to use a VPC as transit network (i.e. routing traffic through a BGP AS). Which is understandable as the last thing Amazon need is customers causing routing loops within their cloud environment. To get around this issue, you'll want to use what's called a Transit VPC.
This VPC functions as a hub for both VPC's and outside networks to route traffic to and from each other. Two Cisco ISR's (1000v) function as the back bone for this VPC. These two virtual routers are used for VPN termination, routing, and high availability. From what I understand these Cisco routers have most of the traditional Cisco IOS XE feature set. So maybe you can get creative with using DMVPN, FlexVPN, etc. for additionally dynamic capabilities.
Like anything Cisco though, you do have to pay a premium for this service. However it appears that this is not only the best choice but probably the easiest to implement.
Have you ran into crazy routing scenarios you've had to get around in a cloud or hybrid environment? Would love to hear your war stories and solutions in the comments below: