Namespaces in My Emacs? It's More Likely than you think!
Implementing namespaces and private symbols using lexical closures.

This is the macro itself and

<<ns-private?>>
<<ns-find-private-refs>>
<<ns-transform-private-refs>>
<<ns-create-private-binds>>
<<ns-remove-private-defs>>

(defmacro ns (namespace &rest forms-and-doc)
  (declare (doc-string 2)
           (indent defun))
  (let* ((forms (if (eq (-> forms-and-doc first type-of) 'string)
                    (rest forms-and-doc)
                  forms-and-doc))
         (private-refs (ns-find-private-refs namespace forms))
         (expanded-forms (-map 'macroexpand-all forms))
         (transformed-forms (ns-transform-private-refs namespace private-refs expanded-forms)))
    `(progn
       (setq lexical-binding t)
       (let ,(ns-create-private-binds namespace transformed-forms)
         ,@(ns-remove-private-defs namespace transformed-forms)))))

To start off everything we search for definitions, value setting, and other similar things to ensure the values are captured in the current namespace. Currently this is only done on the top level because dealing with internal functions gets rather irritating rather quickly. For the purposes of this definition search only defvar, defconst, defmacro, defun, and defalias will be counted.

Now to make a symbol public you simply prefix it with the symbol used in the namespace. So to define a public function one might write:

(ns namespaced
  "Another simple namespacing example."
  (defun private-fn (a b) (+ a b))
  (defun namespaced-public-fn () (private-fn 3 4)))

Thus we also filter out the definitions that begin with the same symbol as the namespace itself. This enforces a standardized approach to function naming and, as long as no two namespaces have the same name, eliminates the risk of name collisions.

Once we have a list of private definitions, we need to transform regular calls into the funcall syntax and unquote quoted names of our functions since our function symbols are actually variables. Now Emacs Lisp is a functional programming language, but the way the functions are defined is through the fset special form rather than simply setting a variable to a lambda, which gives their symbols a different syntax than that of a lambda simply bound to a symbol.

Lexical binding is not active by default and is buffer-local. This is actually pretty bad as it basically causes the lexical closures to suddenly disappear in a puff of smoke. I could use the lexical-let or lexical-let* functions in the cl library, which would solve the problem, though they are specified as being deprecated.

Unfortunately, because the let* special form does not produce lexical closures even with lexical-binding set to t, I cannot effectively use it. This means that all let forms are bound at the same time (rather than sequentially) and therefore cannot be called to set private variables or generate private code.

I could write my own let* replacement to bind them sequentially, which would work though also be a bit inelegant and be working around what I think is a bug in Emacs itself (which is therefore something that ought to be fixed in the C source code, not hacked around).

There is also a major issue when dealing with code without lexical binding as in those cases the functions suddenly break as they are no longer lexical closures, and since lexical-binding is buffer-local, that kind of messes up a lot of code. Therefore I think the next step is to take a deep dive into Emacs Lisp code and look at how I could make lexical binding the default for all new files and the user environment.

Another interesting possibility with namespaces is to drop the whole lexical closure thing instead make them macros that provide access to a list of lambdas indexed by the symbol provided. So for example:

(ns foo
  (defun bar (a) (! (+ a 3))))

Would produce a macro called "foo" that would, depending on it's arguments, expand to a variety of different functions. Then you could call the functions in it like so:

(foo bar 33)

You could also alias that function using the namespace macros produced, which would make it intuitive to directly use your functions in the new namespace while also not requiring lexical scoping.

(defalias (foo bar))

I might try it out sometime soon and will link to the blog post, regardless of how successful the implementation is.

One interesting thing that I found while implementing this is that namespaces are in essence singlton static classes, and that it is really more proper to say that classes are the general case of namespaces. This actually makes me respect the python approach to namespaces a little more.

However, I think the affordances of namespaces and classes are rather different, producing different outcomes in terms of software design. Classes make it easy to envision coupling state with data (sort of like closures), but encourage it rather strongly, often being seen as a extension of the struct concept in C.

Namespaces however, do not obviously make it so that you can couple state and data even if you can use them as singleton classes. Therefore one is inclined to approach them from a more functional perspective.

It also became apparent that you can fairly easily implement all object oriented programming concepts including inheritance (contrary as to what is implied with Let Over Lambda) with lexical closures and lambdas. One simply makes it such that the closure will evaluate another closure within it's environment and then returns that closure, creating a sort of tower of lexical closures.

It is possible to do this all without let, assuming you permit a modification of how lambda works. Instead of having lambda merely producing an anonymous function, imagine if it creates a new lexical scope and that a set function exists that can bind variables within that scope. Using this it is trivial to construct a let function using set and lambda alone. This has pretty much no practical application, though I thought the idea was somewhat cool.