How to Programmatically Create Self-signed Certificate and Key PairAssociation for SSL Communication with Microsoft Windows SSPI

In late 2001 when I started the development of the DICE, a multi-protocol network server, one of the planned features was secure authenticated connection across the web for remote server administration. I implemented it with SSPI (Security Support Provider Interface Architecture) found in Microsoft Platform SDK. SSPI is an abstraction framework through which you can control 3 (or more) different secure authentication/communication protocols including SSL (Secure Sockets Layer).

Among the protocols supported in SSPI, I chose SSL because others (NTLM and Kerberos) were useless in my context over the internet without ActiveDirectory and related mess. But SSL in SSPI has some caveats before use - Since SSL is an inefficient streamed protocol unlike others and the abstraction by SSPI is not in high-level, your code starts to look nasty if you attempt to make it conform to the streaming nature of the protocol. It gets worse especially when your application is constructed around asynchronous sockets. Besides, you need a server certificate prior to negotiation.

Though I chose SSPI, an Open Source solution for SSL is also publically available. OpenSSL, the Open Source toolkit for SSL/TLS, is what people instantly bring up when there's need of SSL in the OSS world. However it's tightly united with the BSD socket (the situation may be changed as of now though) and not straightforwardly usable in my application that is fundamentally based on asyncronous socket connections. Eventually I could use OpenSSL in an unexpected place - certificate generation. My original intention was to let my application generate a server certificate programmatically, but I couldn't because of the restriction in my time and programming skill back then. Using some of OpenSSL APIs in my application to generate a certificate also slipped my mind and I didn't so much as bother to see if it's possible or not. I passed that task over to a user by bundling a simple certificate generator in my application package: a Windows executable of OpenSSL and a batch script to run it. This solution was far from smart and had irritated me for a long time. After I improved other important parts of my application in later versions, I removed all SSPI-related code and replaced SSL with my own simple protocol that rings well with the other parts of the code in terms of space efficiency.

Later in the early summer of 2004 I had some time to consider new features for the forthcomig version of the DICE. One of them was IRC over SSL. Though IRC over SSL is not in the IRC standard and I was not too inclined to implement it with not so delightful experience with SSL as I wrote above, I'd already got a few emails that asked it for the DICE. I browsed SSL-related resources in the MSDN Library for a while and found it looked more accessible than the first time I read it, perhaps because of my ever growing programming skill (ahem). In the following week I implemented IRC over SSL. It was not a trivial task, I had to spend quite some time to investigate why my server couldn't communicate with an IRC client that attempts SSL connection. But finally I could iron out all the problems in SSL handshake and subsequent exchange of encrypted messages - except for certificate generation and how to install a generated certificate in the Windows certificate store. Therefore this document was prepared to tell you how I could solve this little but troublesome problem.

When I made a simple certificate generator with OpenSSL before, it could generate a PFX file that contains a key pair (a public key and a private key) and a certificate associated with the pair. A PFX file is packed in the standard file format called PKCS#12. When you click a file with .pfx extension in the Windows shell (Explorer), it prompts you to install the certificate and the associated key pair in the certificate store of Windows. When your application wants to create an SSL session using SSPI, it has to access the certificate store to retrieve a certificate and a key pair. How can you automate this process by a code without user interaction? If you have enough time to read and understand documents about the PKCS#12 format and the certificate generation algorithm, of course no problem there. If you can embed OpenSSL in your application, it's all right too (you can even learn how OpenSSL creates a PFX file by reading its source code). However, only with Microsoft Platform SDK, how can you tackle this problem?

In my previous attempt, the particular point that prevented me from implementing it in the DICE was, association of a key pair with a certificate in the certificate store. Without this association, an API call to get a credential for SSL authentication fails. Certificate generation doesn't require much effort because in a minute you'll find the useful function verbosely named CertCreateSelfSignCertificate if a self-signed certificate is enough for your use. But there is no straightforward way to associate a key pair with a certificate in the certificate store at least in the MSDN Library. I wrote a test program to create a certificate and install it in the machine certificate store (since my application is a Windows Service it has to use the machine store, not the user store). But everytime I exported an installed certificate from the store, it refused to export a private key with it and I knew I failed.

What I tried next is to browse code samples. The Platform SDK package contains the Samples folder that contains code snippets correspondent to various categories covered by the SDK. What you 'd be interested in for certificate manipulation is found in the Samples\Security\CryptoApi\CreateCert folder where you can locate the CreateCert program source code. What it does is to create a self-signed certificate for testing use. Optionally you can set various properties of a certificate and put it into the certificate stores. I compiled & linked the CreateCert and could run it to do my work... NOT. It lacked one particular feature - it couldn't put a private key of a key pair in a certificate store. When you try to import a generated self-signed certificate and a key pair into a certificate store, a private key is always lost in the process and you can't export it with the certificate later. It means you can't use the certificate imported by the program as a certificate for SSL communication. In other words without your own private key you can't decrypt a ciphertext encrypted with your public key in an SSL channel. At the end of the program it looks like this

          // Set Certificate's Key Provider info
          bResult = CertSetCertificateContextProperty(
              pCertContext,
              CERT_KEY_PROV_INFO_PROP_ID,
              0,
              (LPVOID)&CryptKeyProvInfo
          );

but it's useless in this problem about a private key. This call to CertSetCertificateContextProperty comes after it places a certificate in a certificate store. Apparently what it has to do is associate a private key with the certificate in the store where it puts the key a moment before, so this is the place where you should do something. However, the CreateCert program doesn't have any clue on it.

Fortunately what Microsoft offer you doesn't end there. You have another tool to manipulate a certificate, but not in the Samples folder. Instead, it's in the Bin folder, and called makecert.exe. Yes, it's a binary executable file and naturally comes without the source code. Nevertheless you can assume it has to be able to do basic tasks in certificate manipulation, such as association of a private key with a certificate stored in a certificate store, since without being able to create a working certificate in a certificate store the purpose of this tool is almost void (OK I'm exaggerating but it's still logical to make a conjecture like this). If you run makecert.exe to generate a self-signed certificate and place it in a certificate with a key pair, it can show you the expected result indeed, unlike the CreateCert program. The imported certificate can be exported complete with its private key if you enter the password. So you can guess something is wrong in the CreateCert source code. The CreateCert program seems to be able to do the same thing as makecert.exe does, but something is definitely missing from it.

This is a typical situation where MSDN and Google are supposed to be your friends. But they couldn't help when I asked my question. I'd assumed the how-to on this problem would be in some FAQ about SSL usage on Windows, but I was wrong. It's as if very few people actually used the certificate-related API in the Microsoft platform. The only place it was mentioned I managed to find was in some messages in the microsoft.public.security.crypto newsgroup. It's the newsgroup where the people responsible for Microsoft security products (sometimes) answer questions about the security-related parts of the Microsoft SDK. I found an MS employee or an MSVP wrote "...then you have to associate the private key with the certificate you imported, but it's a bit tricky." and the message had no hints on how it's tricky nor how you can do it. Yeah, it's tricky I KNOW...

Actually that's all what I could collect from the web and MSDN. If makecert.exe were an Open Source tool I'd had no trouble like that. This time, you have to open it by yourself as if it's Open-Sourced - with a technique of reverse-engineering. Now, as I can't reverse-engineer tools made by Microsoft since it's likely to be prohibited in their licenses, I tell you how you can get what you need all by yourself.

There are a few known debuggers for their powerful reverse-engineering and system-diagnosing capability, among which I recommend OllyDbg as the best free tool available. Download it and unpack it, then run it. Load makecert.exe in OllyDbg by File | Open. In Debug | Arguments, feed OllyDbg with the argument line which should be passed into the debug target, makecert.exe, the argument with which you could generate a certificate and put it in a certificate store with an associated key pair. An example would look like

-sr LocalMachine -ss MY -sk Temp000 -n CN=testcert -r

With this argument, it'll create a new key pair and a new certificate, then will place them intertwined in the machine certificate store named 'MY'. 'MY' is the name of the personal store you can find in the certificate MMC. In my case, I wanted to use a certificate and its associated key pair from a Windows service which runs in the SYSTEM account. I had to put it in the local machine store which can be globally seen from all users who belong to the same machine. By setting the argument in OllyDbg, you are prompted to restart the debug target to apply the new argument, so restart it via Debug | Restart. It pauses at the entry point of the target program just like the first time you load the target.

Now all necessary things are laid out before your eyes; you are ready to investigate how makecert.exe works. While you can step through the assembly code all the way, this particular case allows you to take a shortcut. You have already seen the source code of the CreateCert program, which means you already know how the flow chart of the program will look like when you generate a certificate. Control points in this imaginary flow chart are most likely to be mapped to API calls in the actual makecert.exe.

OllyDbg has many convenient features for inspection into internals of a binary code. Focus the CPU window and open the right-click context menu, select Search for | All intermodullar calls. A window that shows all intermodular calls with verbose symbol names will appear. If the destination is kernel32.CreateFileA, it means makecert.exe calls the CreateFileA function in the kernel32.dll kernel module. Now browse the list of the destination functions. Among function calls to kernel32, msvcrt, user32 and others, you'll find many calls to functions in the CRYPT32 module. Among them there'd be ones with the 'Cert' prefix. The primary target is, as you can guess by the CreateCert sample program, CertSetCertificateContextProperty, so look for it.

If you can find it, what you should know next is in what context it's called in makecert.exe. That is, you have to know the correct arguments for the function. OllyDbg can set a breakpoint on every call to a function you choose, so highlight the CertSetCertificateContextProperty, right-click to open the context-menu, select the command "Set breakpoint on every call to CertSetCertificateContextProperty". In addition to this particular function, you may want to add breakpoints to other related functions such as CertAddCertificateContextToStore to manage a certificate store to get additional clues in the flow.

Now let's run the program to reach the place where CertSetCertificateContextProperty is called. Select Debug | Run in the OllyDbg menu. It starts execution of makecert.exe under the debugger and it'll stop at the first time CertSetCertificateContextProperty is called in the program. When it stops at a call to CertSetCertificateContextProperty, you are at a step of a CALL opcode to call the function. Select Debug | Step Into to proceed just 1 step ahead. Now you jump in the block of the function code. Well, what to do here? Of course you've come here to figure out the correct arguments for the function. Select View | Call stack in the menu. You have the window that shows the call stack of the thread. At the top of the stack, there should be a call to CertSetCertificateContextProperty and you can see what arguments are actually set in the call. According to the MSDN Library, the signature of CertSetCertificateContextProperty is like this:

BOOL WINAPI CertSetCertificateContextProperty(
  PCCERT_CONTEXT pCertContext,
  DWORD dwPropId,
  DWORD dwFlags,
  const void* pvData
);

The first argument is a handle to a certificate you fiddle with. The second one, "Arg2" in a call stack window of OllyDbg, is used to specify which property of a certificate you are interested in. If it's 0x00000002, see wincrypt.h header file in the Platform SDK and you'll know CERT_KEY_PROV_INFO_PROP_ID is defined as 0x00000002. CERT_KEY_PROV_INFO_PROP_ID is, as its name suggests, a property to store info about a key provider. The fourth argument is where you set actual data to pass as a new property value. If Win32 API were organized in an object-oriented way it'd be far easier to manage those certificates, but in reality you have to control those objects by C functions mimicking OO in a not-so-intuitive way. Anyway, the fourth argument should be a pointer to a CRYPT_KEY_PROV_INFO struct. OllyDbg is nifty to track these arguments, the only thing what you have to do manually is highlight the argument in the call stack window and select "Follow address in stack" in the right-click context menu. The CPU window is raised up, in the right bottom of it you'll see a memory map around the struct passed to the function. The CRYPT_KEY_PROV_INFO struct is defined in wincrypt.h like this:

typedef struct _CRYPT_KEY_PROV_INFO {
    LPWSTR                  pwszContainerName;
    LPWSTR                  pwszProvName;
    DWORD                   dwProvType;
    DWORD                   dwFlags;
    DWORD                   cProvParam;
    PCRYPT_KEY_PROV_PARAM   rgProvParam;
    DWORD                   dwKeySpec;
} CRYPT_KEY_PROV_INFO, *PCRYPT_KEY_PROV_INFO;

The memory layout is: a 32-bit/4 bytes (if you are on Win32) pointer to a wide string for a container name, another 4-bytes pointer to a provider name string, a 4-bytes DWORD for a provider type, another 4-bytes DWORD for flags, and the rest. OllyDbg can show actual UNICODE strings pointed by those LPWSTR pointers. The container name you set in the argument for makecert.exe should be there. (In the aforementioned example, "Temp000".) The memory map shown in OllyDbg is stacked as 32-bit blocks. Just under the container name you can see a provider name such as "Microsoft Strong Cryptographic Provider" found in wincrypt.h as the value of MS_DEF_PROV. The block below the pointer to a provider name will be the value 0x00000001. It's the value of PROV_RSA_FULL provider type. The next block is for flags and it's what I couldn't make out only from the description in the MSDN Library.

The document in the MSDN Library mentions CryptAcquireContext flags like "See CryptAcquireContext for the list of flags passed through." but only lists 2 special flags on the page, and the function I intended to use was CertSetCertificateContextProperty and not CryptAcquireContext. Thus I had no idea that there are other flags. What's unlucky for me was the CreateCert sample code is completely lacking in this point. Basically it sets nothing in the flags and Microsoft leaves you in the cold when you try to associate a private key with a certificate.

Now watch what value is in the place of dwFlags - makecert.exe sets 0x00000020 in dwFlags of CRYPT_KEY_PROV_INFO struct. In wincrypt.h, it's the value of CRYPT_MACHINE_KEYSET. Yes, it's plain and simple (to the shame of me who couldn't make out it at the first glance in the MSDN Library document). The correct key container is selected by setting the value in dwFlags when a private key is associated with a newly created certificate. Since the CreateCert sample code doesn't set it, a private key is lost and you can't get a credential handle by the AcquireCredentialsHandle API.

From here on I'll show you a complete C++ sample code to programmatically generate a PKCS#12-formatted PFX file only with the Windows Platform SDK functions. When built and executed, it generates a self-signed certificate and an associated keypair and puts them in the machine certificate store.

First we put the necessary headers...

// for FXExportCertStoreEx
#ifndef _WIN32_WINNT
#define _WIN32_WINNT 0x0500
#endif

// some certificate functions are affinitive with UNICODE
#ifndef UNICODE
#define UNICODE
#endif

// required to link this library for certificate functions
#pragma comment(lib, "Crypt32.lib")

#include <windows.h>
#include <wincrypt.h>
#include <malloc.h>

#include <iostream>
#include <fstream>

using namespace std;

C++ headers iostream and fstream are just included for the sake of easier file access.

Let's proceed to the main function. This simple program has no options as they are hardcoded as variables such as a certificate subject name and a key container name. "CN" stands for common name in the X.500 standard.

int main()
{
 wchar_t* pszCertificateSubjectName = L"CN=Test Subject";

 DWORD dwSize = 0;
 if (!CertStrToName(
  X509_ASN_ENCODING,
  pszCertificateSubjectName,
  CERT_OID_NAME_STR,
  NULL,
  NULL,
  &dwSize,
  NULL
 ))
 {
  cerr << "Invalid certificate subject name" << endl;
  return 1;
 }

 PBYTE p = (PBYTE)_alloca(dwSize);

 if (!CertStrToName(
  X509_ASN_ENCODING,
  pszCertificateSubjectName,
  CERT_OID_NAME_STR,
  NULL,
  p,
  &dwSize,
  NULL
 ))
 {
  cerr << "Invalid certificate subject name" << endl;
  return 1;
 }

This apparently redundant operation is typical for crypto functions. You have to encode a plain string into a crypto-friendly format by youself to pass it to a crypt function. Memory management is all up to you, so these APIs don't have convenient features such as automatic conversion. Since the correct size of an encoded string is not known beforehand, you have to call a conversion API to know the final size, allocate a memory, then set it as the destination of a conversion.

Here we prepare a key container in the RSA cryptographic service provider for generation and verification of a key-pair and a certificate by the RSA public key algorithm which is now patent-free.

 CERT_NAME_BLOB sib;
 sib.cbData = dwSize;
 sib.pbData = p;

 wchar_t* pszKeyContainerName = L"Test Container Name";

 HCRYPTPROV hProv = NULL;
 if (!CryptAcquireContext(
  &hProv,
  pszKeyContainerName,
  MS_DEF_PROV,
  PROV_RSA_FULL,
  CRYPT_NEWKEYSET | CRYPT_MACHINE_KEYSET
 ))
 {
  if (GetLastError() == NTE_EXISTS)
  {
   if (!CryptAcquireContext(
    &hProv,
    pszKeyContainerName,
    MS_DEF_PROV,
    PROV_RSA_FULL,
    CRYPT_MACHINE_KEYSET
   ))
   {
    cerr << "Can't get a crypto provider" << endl;
    return 1;
   }
  }
 }

Next, you have to open the "MY" certificate store in the local machine. The "MY" certificate store is where personal certificates are stored. Then, if it contains a certificate previously generated by this program, the previous one is deleted.

 HANDLE hCertStore = CertOpenStore(
  CERT_STORE_PROV_SYSTEM,
  X509_ASN_ENCODING | PKCS_7_ASN_ENCODING,
  hProv,
  CERT_SYSTEM_STORE_LOCAL_MACHINE
   | CERT_STORE_NO_CRYPT_RELEASE_FLAG
   | CERT_STORE_OPEN_EXISTING_FLAG,
  L"MY"
 );
 if (!hCertStore)
 {
  CryptReleaseContext(hProv, 0);
  cerr << "Can't open the \"MY\" certificate store in this local machine" << endl;
  return 1;
 }

 PCCERT_CONTEXT pCertContext = CertFindCertificateInStore(
  hCertStore,
  X509_ASN_ENCODING | PKCS_7_ASN_ENCODING,
  0,
  CERT_FIND_SUBJECT_NAME,
  &sib,
  NULL
 );
 if (pCertContext)
 {
  cerr << "Found the certificate - deleting it" << endl;
  if (!CertDeleteCertificateFromStore(pCertContext))
  {
   CryptReleaseContext(hProv, 0);
   cerr << "Can't remove the old certificate" << endl;
   return 1;
  }

  pCertContext = NULL;
 }

Now that the certificate store is ready, let's generate a new key pair for key exchange in the key container by the CryptGenKey function with AT_KEYEXCHANGE as it's used for SSL encrypted communication. Then, set up a struct with properties you've prepared so far and call CertCreateSelfSignCertificate with it to create a self-signed certificate.

 HCRYPTKEY hKey;
 if (!CryptGenKey(hProv, AT_KEYEXCHANGE, CRYPT_EXPORTABLE, &hKey))
 {
  CryptReleaseContext(hProv, 0);
  cerr << "Can't generate a key pair" << endl;
  return 1;
 }

 CRYPT_KEY_PROV_INFO kpi;
 ZeroMemory(&kpi, sizeof(kpi));
 kpi.pwszContainerName = pszKeyContainerName;
 kpi.pwszProvName = MS_DEF_PROV;
 kpi.dwProvType = PROV_RSA_FULL;
 kpi.dwFlags = CERT_SET_KEY_CONTEXT_PROP_ID;
 kpi.dwKeySpec = AT_KEYEXCHANGE;

 SYSTEMTIME et;
 GetSystemTime(&et);
 et.wYear += 10;

 CERT_EXTENSIONS exts;
 ZeroMemory(&exts, sizeof(exts));

 PCCERT_CONTEXT pc = CertCreateSelfSignCertificate(
  hProv,
  &sib,
  0,
  &kpi,
  NULL,
  NULL,
  &et,
  &exts
 );
 if (!pc)
 {
  CryptDestroyKey(hKey);
  CryptReleaseContext(hProv, 0);
  cerr << "Can't create a self-signed certificate" << endl;
  return 1;
 }

A newly created certificate comes as a certificate context. You have to put it in the certificate store prepared in the previous steps.

 if (!CertAddCertificateContextToStore(
  hCertStore,
  pc,
  CERT_STORE_ADD_REPLACE_EXISTING,
  &pCertContext
 ))
 {
  CryptDestroyKey(hKey);
  CertFreeCertificateContext(pc);
  CryptReleaseContext(hProv, 0);
  cerr << "Can't create a self-signed certificate" << endl;
  return 1;
 }

 CertFreeCertificateContext(pc);

...and this is the most important point of this whole article. It shows how you associate a private key with a certificate in a certificate store. Note that it gives pCertContext, instead of pc, to the CertSetCertificateContextProperty function. pCertContext is the certificate context you get as a handle to the certificate you just put in the certificate store. Since it's a handle stored in the certificate store, any operation to the handle affects the certificate in the store. By setting CRYPT_MACHINE_KEYSET in the flags, it searches a correct place for the key container and sets a new property value in it.

 CRYPT_KEY_PROV_INFO ckp;
 ZeroMemory(&ckp, sizeof(ckp));
 ckp.pwszContainerName = pszKeyContainerName;
 ckp.pwszProvName = MS_DEF_PROV;
 ckp.dwProvType = PROV_RSA_FULL;
 ckp.dwFlags = CRYPT_MACHINE_KEYSET;
 ckp.dwKeySpec = AT_KEYEXCHANGE;

 if (!CertSetCertificateContextProperty(
  pCertContext,
  CERT_KEY_PROV_INFO_PROP_ID,
  0,
  &ckp
 ))
 {
  CertFreeCertificateContext(pCertContext);
  pCertContext = NULL;
  CryptDestroyKey(hKey);
  CryptReleaseContext(hProv, 0);
  cerr << "Can't set certificate property" << endl;
  return 1;
 }

The certificate and the key pair are ready in the store, so let's try to export them in one as a PKCS#12 PFX file. For this operation the platform SDK has very straight PFX-managing functions so you won't have any trouble. All you have to do is to save a PFX binary blob as a file.

 CRYPT_DATA_BLOB cdb;
 ZeroMemory(&cdb, sizeof(cdb));

 wchar_t* pszPassword = L"Test Password";

 if (!PFXExportCertStoreEx(
  hCertStore,
  &cdb,
  pszPassword,
  NULL,
  EXPORT_PRIVATE_KEYS
 ) || cdb.cbData == 0)
 {
  CertFreeCertificateContext(pCertContext);
  pCertContext = NULL;
  CryptDestroyKey(hKey);
  CryptReleaseContext(hProv, 0);
  cerr << "Can't export a certificate from the certificate store" << endl;
  return 1;
 }

 cdb.pbData = (PBYTE)_alloca(cdb.cbData);

 if (!PFXExportCertStoreEx(
  hCertStore,
  &cdb,
  pszPassword,
  NULL,
  EXPORT_PRIVATE_KEYS
 ) || cdb.cbData == 0)
 {
  CertFreeCertificateContext(pCertContext);
  pCertContext = NULL;
  CryptDestroyKey(hKey);
  CryptReleaseContext(hProv, 0);
  cerr << "Can't export a certificate from the certificate store" << endl;
  return 1;
 }

 {
  ofstream out("test.pfx", ios::binary);
  if (!out)
  {
   CertFreeCertificateContext(pCertContext);
   pCertContext = NULL;
   CryptDestroyKey(hKey);
   CryptReleaseContext(hProv, 0);
   cerr << "Can't write a pfx blob" << endl;
   return 1;
  }

  out.write((char*)cdb.pbData, cdb.cbData);

  if (!out)
  {
   CertFreeCertificateContext(pCertContext);
   pCertContext = NULL;
   CryptDestroyKey(hKey);
   CryptReleaseContext(hProv, 0);
   cerr << "Can't write a pfx blob" << endl;
   return 1;
  }
 }

 CertFreeCertificateContext(pCertContext);
 pCertContext = NULL;
 CryptDestroyKey(hKey);
 CryptReleaseContext(hProv, 0);

 cout << "Done" << endl;

 return 0;
}

Since Microsoft pushs .NET Framework for daily work these days, those complicated native Windows APIs are not as often employed as before. But still the fact that some arcane cryptographic functions can't find their peers in .NET Framework forces you to use APIs in the Platform SDK if necessary. Performance is another reason not to rely on .NET Framework where some methods are internally implemented in Managed code as seen in Rijndael encryption in the System.Security.Cryptography namespace. If Open Source libraries are available you are free to use them, but using Platform SDK can suppress bloat in executable size. It's a trade-off after all, however you have no choice sometimes, just to the degree where you have to resort to things like reverse-engineering.