That is true when you are instance pooling. (Stateful or Stateless)
From memory; if you are not instance pooling, the component will go away on SetComplete() or SetAbort(). The component should go away on client disconnect, but if you are untidy here this might not be the case. Hence the original question.
So the answer to the original question should perhaps be:
For a stateful component create an exposed function containing the SetComplete() statement, This will trigger the deactivate event where you place clean up. The component is then released to the instance pool or destroyed depending on component properties.