I know there are plug-ins available, but is there no way to just set up iCloud contacts as an external data source? I'm not averse to getting my hands dirty with a little AppleScriptObjC or even Swift if need be.
The plugins are the easiest way to go.
As for making a direct, external data source connection to an iCloud contacts database... Well, it's not likely. It doesn't have a native ODBC driver for it. So anything you would have to do in order to establish an ODBC connection, would be much more difficult then just using a plugin.
iCloud does have the CloudKit API that you could use:
CloudKit Web Services Reference: Fetching Contacts (users/lookup/contacts)
If you don't actually need it as an external data source and just want to CRUD between filemaker and iCloud, then this may be an easier way to go; Especially with all the recent additions in FM16 to handle complex API calls via Insert From URL.
Why the adversion to a plugin?
Cost? I bet the time savings is worth the cost?
Retrieving data ...