MS SQL DBA here... "Hides the database files in it's own proprietary format" sounds a little off.
SQL Server can use anything Windows can use as a datasource. So you can have SQL Server making queries against a plain text file joined against an excel spreadsheet, joined against data is mysql, or sqlite. This is one of the strengths of SQL Server, it can wire in almost anything.
SQL Server also has MANY options for encryption of the underlying files.
My suspicion is that SQL Server is either A) connecting to a well known file type that it supports as a linked server / linked data source which is something you didn't realize it can connect to, or B) using a regular SQL Server file but using the encryption options to ensure that the data is secure at rest.
I would tend toward the latter B). If you don't want people to be able to snag the database file and dump the data, SQL Server encryption will handle that easily. No need to create a special file format. A SQL Server developer would know that and go that route first.