The pagination is defined in the template/catalog/product/list/toolbar.phtml where you can see getPagerHtml() function for the pagination. But you cannot modify pagination style from here. There is template file template/page/html/pager.phtml where you can make changes.

There is option in Magento Admin. To change from admin, you have to login to Magento admin and nevigate to:

System -> configuration -> General -> Design -> pagination

And change one of following values.

Field Description
Pagination Frame The number of links to display at one time.
Pagination Frame Skip If the current frame position does not cover utmost pages, it will render link to current position plus/minus the value.
Anchor Text for Previous Alternative text for the previous link in the pagination menu. If this field is left blank, a default arrow image is used.
Anchor Text for Next Alternative text for the next link in the pagination menu. If this field is left blank, a default arrow image is used.