include functor

[ccasin]

This is a proposal for a new structure and signature item form, include functor.

Rendered version

(Thanks to @OlivierNicole and @goldfirere for help preparing this RFC)

[samsa1]

In a similar way could we also add a module functor E = F

This might be useful in some other cases and does not seem much work in addition to this feature. The use case that I have in mind would be linked to modular implicits :

module type Eq = sig
    type t
    val eq : t -> t -> bool
end

module type Ord = sig
    type t
    val cmp : t -> t -> int
    module E : Eq with type t = t
end

module F (X : sig type t val ord : t -> t -> int) : Ord with type t = X.t = body

module OInt = struct
    type t = int
    let cmp = Int.compare
    module functor E = F
end

This allows for the OInt module (an instance of Ord) to automatically implement an equality that respects ordering.

[ccasin]

In a similar way could we also add a module functor E = F

Indeed. I agree this is a reasonable feature and not much more work. It has occasionally been requested by users of include functor at Jane Street. We've held off on implementing it, but not for any particularly principled reason (mainly: I think it will get a little less use, and I think the meaning of the syntax is slightly less intuitive) but I'm very happy to add it if there is consensus it is desirable.

[chambart]

This is a pattern that is actually quite common in the flambda2 code base, in particular

module T = struct
  module M = struct
    type t = ...
    let compare = ...
  end
  include M
  module Set = Set.Make(M)
end

Which would allow to get rid of that spurious M module

module T = struct
  type t = ...
  let compare = ...
  module functor Set = Set.Make
end

One such example in the upstream compiler code base:
https://github.com/ocaml/ocaml/blob/0d18e1287e49e92cf37824559cda5c09a2438b32/typing/shape.ml#L103-L135

[yallop]

Have you considered the alternative of giving a name (say "_") to the current module prefix (i.e. "the module up to this point"), so that instead of

module M = struct
  type t = ...
  [@@deriving compare, sexp]
  include functor Comparable.Make
end

you'd write

module M = struct
  type t = ...
  [@@deriving compare, sexp]
  include Comparable.Make(_)
end

?

With that alternative design it'd be possible to refer to the module prefix in arbitrary module expressions rather than always passing it as the argument of a single-parameter functor, so you could also write things like:

include F(_)(X)

and

module type of _

and

open F(_)

and

module E = F(_)

and

include S with module type T = _

and perhaps even

type t = F(_).t

etc.

[alainfrisch]

whose parameter can be "filled in" with the previous contents of the module

Just to be sure : do the components used to "fill in" the parameter need to be defined from the current structure, or do they only need to be visible at this point (coming from a surrounding structure or from some open)?

[lthls]

We're not very fond of the underscore, so @Ekdohibs suggests module as of then and @chambart proposes virtual module downto begin

[samsa1]

A better argument against using underscore to talk about the beginning of the module is that current work on modular implicits. We are currently thinking of defining _ as an arbitrary module expression that should be inferred but this would be incompatible with the proposal of @yallop.
However I think that his idea is more expressive and should be discussed but with another name in mind.

[ccasin]

Just to be sure : do the components used to "fill in" the parameter need to be defined from the current structure, or do they only need to be visible at this point (coming from a surrounding structure or from some open)?

They need to be defined from the current structure. One could imagine doing either thing, but this has a nice clear rule, makes it less likely refactorings will cause errors due to what is in scope for include functor changing, and simplifies the implementation.

[goldfirere]

For syntax, we could use use just plain old module. Examples:

include F(module)
module M = F(module)

Or we could be even bolder and use a symbol:

include F(^^)
module M = F(^^)

I think any syntax should not be available in paths.

[lpw25]

Personally, I dislike both the:

module functor E = F

form and mechanisms based on a name for the contents of the current module, and would prefer to push people towards include functor instead. That is because I think it is better to have a name for this interface:

sig
  type t
  module Set : Set.S with type elt = t
end

and use that, rather than having each user choose the name for their set module.

include functor supports that style very naturally. In the Set module you can define:

module type MixS = (X : OrderedType) -> sig module Set : S with type elt = X.t end
module Mix : MixS

and then you can write:

module Foo : sig
  type t
  include functor Set.MixS
end = struct
  type t = [...]
  let compare = [...]
  include functor Set.Mix
end

[yallop]

If I understand correctly, this use of include functor in signatures amounts to treating functor types as parameterized signatures. It certainly makes the example look elegant, but it doesn't really seem harmonious with the way that module types work in the rest of the language.

[lpw25]

That is one way to look at it and it does look different from other uses of module types in that perspective. An alternative though is to consider include S to mean "extend the module type as it would be if it had include M done to it for some unknown M : S, and then treat include functor S in the same way: extend the module type as it would be if it had include functor M done to it for some unknown M : S. I think that is a quite natural way for users to think about it, and there isn't any other obvious way to interpret include functor S in a signature.

[clementblaudeau]

If I understand correctly, include functor for modules (not for signatures) is needed when the functor does not re-export its parameter. I.e, the pattern

module F = functor (Y:S) -> struct (* ... *) end  
module Foo = struct
   (* code *)
   include functor F
end

could be replaced by changing F to re-export its argument and putting the application at top-level :

module F = functor (Y:S) -> struct include Y (* ... *) end
module Foo = F(struct 
   (* code *)
end)

Overall, could the role of include functor be taken by having a special mechanism to apply and include argument in the result ? A downside I can see is that it puts the functor application at the beginning of the struct, which has not the same flow as putting include functor at the relevant point inside the structure.

[ccasin]

If I understand correctly, include functor for modules (not for signatures) is needed when the functor does not re-export its parameter. I.e, the pattern

module F = functor (Y:S) -> struct (* ... *) end  
module Foo = struct
   (* code *)
   include functor F
end

could be replaced by changing F to re-export its argument and putting the application at top-level :

module F = functor (Y:S) -> struct include Y (* ... *) end
module Foo = F(struct 
   (* code *)
end)

Overall, could the role of include functor be taken by having a special mechanism to apply and include argument in the result ? A downside I can see is that it puts the functor application at the beginning of the struct, which has not the same flow as putting include functor at the relevant point inside the structure.

I think this is a reasonable idea, but doesn't quite offer the full convenience of include functor. In this example from the RFC:

module M = struct
  module T = struct
    type t = ...
    [@@deriving compare, sexp]
  end

  include T
  include Comparable.Make(T)
end

I think your proposal saves the include T, but not the need to define T in the first place when its only purpose is to be a parameter.

[clementblaudeau]

It can save T by doing a functor call directly on the unnamed structure. To be more precise:
What I had in mind was some new construct to mark functor applications where the functor parameter should be included in the result of the application, something like F [reexport] (M) which is syntactic sugar for

struct
  open (struct module X = M end)
  include X
  include F(X)
end

Then the example of the RFC would become:

module M = Comparable.Make [reexport] (struct
  type t = ...
  [@@ deriving compare, sexp]
end)

I think it provides more or less the same functionality. An upside is that it does not depend on a specific position in the code like include functor does, which I think might be a bit brittle. A downside is that it puts the functor application at the top, not in the flow of the definition of the module like include functor does. I'm not sure how it would support patterns where there are several include functors separated by other bindings, like :

module M = struct
  type t = ...
  include functor F
  let x = 42
  include functor G 
end

[clementblaudeau]

Actually a key issue with the re-export pattern I was suggesting is that the functor can only re-export the field indicated in its parameter signature, which seems much more restricted than include functor, for which all fields of the current structure are kept.