I got two computers hooked up to a linksys router.

I was wondering if there was a way to see exactly how much bandwidth each computer is using.

It may export network utilization via SNMP, but I've never used any Linksys equipment for serious work so I don't know their capabilities offhand. MRTG will be able to read the SNMP fields if it handles that, and it's free.

Alternatively you can get a tool like Ethereal (also free) that sniffs all the ethernet frames and can give you a general picture of network traffic as that machine sees it.