I would probably not pay too many points now to get a much lower rate in case rates drop further.
Other things to consider is how stable is your job/income? If it’s not very stable and you can’t refi easily, then maybe it’s worth it to pay points to get a lower rate since a refi may not be in the cards in the future.
You also have to look at your tax situation on whether you can write off the $$ paid to buy down the rate this year (that came up with scaredycat since it’s near year end now and they’re not itemizing)…