Pontivy straddles the river Blavet which runs from the hills of central Brittany down to the sea at Lorient. Canalized all the way and joined to the Nantes Brest canal, this was a major waterway and made Pontivy an important port. The old, medieval town, with its pedestrian streets and open squares, is pleasant to wander around and has some interesting shops and restaurants, as well as a weekly market in the main street.

Pontivy also boasts a castle which was built by the influential Rohan family in the 15th century. Although there is not much of it left, it is nevertheless quite imposing, raised above the end of the main street, and it is open for visits daily.