It's possible via a script as long as your references do not form a circle. If link doc A to Doc B to Doc C and Doc links to Doc A, you'll have a problem.
A script could use recursion to call itself to traverse what is essentially a "Tree" data structure, counting the related records at each level of recursion and returning the count to the calling instance of the script.
What you want might be adapted from the script in this demo file: https://dl.dropboxusercontent.com/u/78737945/RecursiveBOMDemo.fmp12
In this script each item has a Bill of Materials and each item in the Bill of Materials (BOM) can also list a Bill of Materials in a manner very similar to your list of linked documents.
I'll check this out and see if I can make sense of it :-)