It's been a while. Maybe you noticed! I've been working on various odds and ends, and my focus has shifted around a bit. I'll probably talk about this more in the future, but the short version is: Yes, I am still working on Geas but it may be taking a bit of a backseat for a while, and no, I'm not dead, nor have I just stopped doing anything worthwhile in my spare time.

I've been working on a project using LLVM lately. I got it compiling on Windows, I got it linked into my project, and I even got a little LLVM JIT function built and running. It takes in an int32 and returns that number + 1. Hooray.

Problems cropped up with the next step: Making it possible for the VM to call out to my external C functions. In all of the tutorials I could find that discussed doing this, it seems like they declared a function prototype for an external function, and then it just worked. I am here to tell you that, in my environment, developing using Visual C++ on Windows, it does not just work.

After a fair bit of beating my head against various walls and tearing out most of the hair that I have left, I finally figured out how to make it work. I want to record it here so that if anyone else runs into this problem, maybe Google will direct them here and I can help them out.

Here's the important C code...

C++ code
GenericValue gv;
std::vector args;

FunctionType *ft = FunctionType::get(Type::getVoidTy(context), false); 
Function *g = Function::Create(ft, Function::ExternalLinkage, Twine("testCall"), module);
gv = exec->runFunction(g, args);

... and what LLVM tells me was the content of the module ...

LLVM Module output
; ModuleID = 'egg'

declare void @testCall()

... which looks fine. But that exec->runFunction call would crash and burn with the following error message:

LLVM ERROR: Tried to execute an unknown external function: void ()* testCall

It's probably worthwhile to note here that this same thing happened with the version of Kaleidoscope, the toy language from the LLVM tutorials, but only if I tried to import one of the functions defined in the Kaleidoscope code, like putchard() (or so it seemed). I could import say, sin() just fine.

Now, it looks like the reason this works for others is because they have libffi installed, a fancy bit of code that lets you introspect on your compiled code to find other function names at runtime. That sounds great, but let's just say that trying to get it to compile on Windows to work with Visual Studio was not great. So I figured that there must be some way to do this without using ffi, since it seemed to have conditional compile paths for if it wasn't there.

And there is! It's a good thing LLVM is open source — I didn't find the answer anywhere in documentation. I found it by delving through the source code, tracking down the error I was getting, and tracing execution around. Good times.

Anyway, the upshot is that I discovered two important things:

1. LLVM actually inserts some common library functions (sin, printf, etc.) into your externals for you, no matter what. This is why Kaleidoscope was able to extern in things like sin() but not putchard().
2. To manually insert a symbol into LLVM's interpreter, you need to use sys::DynamicLibrary::AddSymbol(), and you need to write a wrapper function.

GenericValue lle_X_FUNCTIONNAME(FunctionType *ft, const std::vector &args)

sys::DynamicLibrary::AddSymbol("lle_X_FUNCTIONNAME", (void *)lle_X_FUNCTIONNAME);

Code is from my project, hasn't been tested in this form, so YMMV.

#include "llvm/LLVMContext.h"
#include "llvm/Module.h"
#include "llvm/DerivedTypes.h"
#include "llvm/Constants.h"
#include "llvm/Instructions.h"
#include "llvm/ModuleProvider.h"
#include "llvm/Analysis/Verifier.h"
#include "llvm/ExecutionEngine/ExecutionEngine.h"
#include "llvm/ExecutionEngine/Interpreter.h"
#include "llvm/ExecutionEngine/JIT.h"
#include "llvm/ExecutionEngine/GenericValue.h"
#include "llvm/Support/raw_ostream.h"
#include "llvm/System/DynamicLibrary.h"

using namespace llvm;

GenericValue lle_X_testCall(FunctionType *ft, const std::vector &args) {
	Log::debug("testCall wrapper has executed.");

	GenericValue gv;
	gv.IntVal = 0;
	return gv;
}

int main(int argc, char **argv) {
	LLVMContext context;
	Module *module = new Module("egg", context);
	ExecutionEngine *exec = EngineBuilder(module).create();

	FunctionType *ft = FunctionType::get(Type::getVoidTy(context), false); 
	Function *g = Function::Create(ft, Function::ExternalLinkage, Twine("testCall"), module);
	sys::DynamicLibrary::AddSymbol("lle_X_testCall", (void *)lle_X_testCall);

	GenericValue gv;
	std::vector args;

	args.clear();
	gv = exec->runFunction(g, args);
}