So as a frequent internet shopper, I very rarely buy something that I plan on returning, but sometimes, it happens, and sometimes, things are out of stock. Why is it that a retailer is able to charge you right away, even if the system knows that the part is out of stock, but it can take a week or sometimes longer to put the money back onto your card? Am I the only one that this bothers?

If you return something to a store, you almost instantly get your money back, either in cash or as a credit. Why can't internet retailers do the same?
The primary reason is because the credit card processing on an internet transaction is quiet different than on a retail transaction. When you place an order on the internet for an item that is out of stock, theoretically you are buying an item that is on back order and as soon as they receive their new stock of items, you are going to be on their queue. Now, when you return an item that you bought on the internet, the store needs to verify that the item was return and restocked. The verification process could take up to a week (give or take 2-3 days returned shipping) and since your order will not be the only return that they got during the day, it could take them a couple of more days to verify the return. As soon as they check that they have received your returned item, they will credit your account back. At a retail store, you are returning the items to an associate and a store and since they can verify that in front of you, the credit is issued back to your account instantaneously. Hope this helps!
Because there is a VERY common internet scam related to that type of activity.