Singular: That chair that has a cushion is comfortable.
Plural: Those chairs that have cushions are confortable.
However the word "that" in the sentence you gave is an introductory word to introduce the relative clause "have cushions" if it's the "that" you referred to. In this case, it indeed doesn't have any other form when serving as an introductory word for clauses.
If you have a question about the English language and would like to ask one of our many English teachers and language experts, please click the button below to let us know: