I use the filter that is part of my fridge for any cold water use (change filter out occasionally, maybe not on the exact schedule but maybe every 6 months or so).
Any hot water use (tea, coffee, cooking etc..) I just use the tap water, figure most bad stuff gets boiled away.