Are Language Models Actually Useful for Time Series Forecasting?