Must I have my operating system date format is set to MM/DD/YYYY?
No, but you have to be sure that StartDay is a REAL date field ( not a text field )
I found a VERY strange solution: using the function Month(StartDay) as it should be used, but in the layout I formatted the result (a date field) to show only the first 2 digits of the DAY (!). Now it works.
Formatting has no effect upon calculations.
As raybaudi asked, is the field StartDay defined as a date field? Note that the Help example used literal text. If StartDate is a true date field and is entered in the format consistent with the system settings (ie: dd/mm/yyyy in your case, presumably) then it will correctly return the month number.
Both fields are proper date fields. The date input in the StartDay field is done with the calendar at end of the field. If the result field is not formatted as mentioned, a date 16.01.2012 will show as 1.1.0001, instead as a month number.
But the Month calculation should be set to return the result type 'Number'. What you are showing is that you have set the MonthNumber to be a 'Date' also, so it is displaying the number 1 as the first day after 00/01/0001.
(Hence your result can never be more than 12, so will never make its way beyond the middle of January, apparently...)
Ok, changing it to a number field solved the puzzle for me.