All versions of OS X thru 10.6 used an open source Samba based version of SMB, which has been considered rock solid for years.
In 2011 Apple ran into problems with Samba GPLv3 licensing requirements, and they they weren’t able to use this Samba based SMB protocol in OS X 10.7 or 10.8. They replaced it with an Apple homebrew version of SMB that they called SMBX, which was developed to be compatible with SMB1 and SMB2 protocols.
In my opinion, SMBX never lived up to the old Samba based SMB protocol, and was buggy at best… especially if you tried to host files via SMB on an OS X 10.7 or 10.8 server.
According to Apple’s OS X Mavericks Core Technology Overview, OS X 10.9 will introduce the SMB2 protocol (a.k.a. CIFS). This could be a very good thing, because SMB2 is well established in the Windows world, and also has excellent track record with Linux. I’m hoping this will alleviate many of the SMB network issues introduced by OS X 10.7 and 10.8.
Apple is stating that “SMB2 is the new default protocol for sharing files in OS X Mavericks”. Based on my research, I believe this means that if you turn on file sharing it will likely only use SMB2, although it’s possible AFP sharing will be an available option (similar to the way SMB sharing is an available option with OS X 10.7 and 10.8). It is unknown if OS X 10.9 Server will include the ability to host files via AFP.
AFP is not however being removed from OS X Mavericks… which is reported to maintain “support for AFP and SMB network file-sharing protocols, automatically selecting the appropriate protocol” and include “Built-in AFP support to provide connectivity with older Mac computers and Time Machine–based backup systems”.