This is sometimes correct. It depends on the storage options of the calculation field. If it is set as unstored, the field does not evaluate until it is referenced by another calculation or displayed on a layout. Some calculations cannot be defined as "stored" as they reference a global or summary field value. Some other calculations can be defined as stored, but will fail to update properly unless set as unstored--this is the case when the calculation does not refer to a field in the same record such as Get ( CurrentDate ).
If it is a stored calculation or a data field with an auto-entered calculation, then your description is correct. You can have any number of data and (stored) calculation fields hidden away as part of the same record but that are not present on the current layout.