I've been trying to rangle the problem of programming language package/modules/etc. I'm not sure how to handle them in Odin but the current approach of having every file be a scope is probably not a good idea in the long run. I did want people to write single file libraries as the norm because this just means you can drop a file in and don't worry about anything else. However, for bigger projects, having multiple files for a single "package" is very useful. So the problem I have is determining what is the best approach to handling multiple file packages, if they are necessarily, and if single file libraries are possible along side multiple file packages.
To me, a multiple file package would have to consist of a directory with source files in. The best approach to this that I have seen is Go(lang). There are many other problems to Go (such as absolute paths) but the general idea of the "directory is the package" is okay. Its approach is borrowed slightly from Modula.
The other approach is the Python approach and still have the file be a scope but allow for a special file in a directory (__init__.py) to treat the directory as if it was a file. This problem does mean that the author of the library can manually organize what they want to be exported or not my explicit declaring it in the __init__.py file. However, if the user really wants to use the internals, they can.
So my question to everyone here is, is the ideal of single file libraries even a problem outside of C and C++ and is passing around a directory contain the files okay, even if it a single file? There is no perfect solution to this problem, only trade-offs.
If anyone has a preferred approach to this problem, I would love to know as I just cannot find an approach I like.
n.b. I am literally nearing feature completion in Odin and I just want to tackle the final big problems
To me, a multiple file package would have to consist of a directory with source files in. The best approach to this that I have seen is Go(lang). There are many other problems to Go (such as absolute paths) but the general idea of the "directory is the package" is okay. Its approach is borrowed slightly from Modula.
The other approach is the Python approach and still have the file be a scope but allow for a special file in a directory (__init__.py) to treat the directory as if it was a file. This problem does mean that the author of the library can manually organize what they want to be exported or not my explicit declaring it in the __init__.py file. However, if the user really wants to use the internals, they can.
So my question to everyone here is, is the ideal of single file libraries even a problem outside of C and C++ and is passing around a directory contain the files okay, even if it a single file? There is no perfect solution to this problem, only trade-offs.
If anyone has a preferred approach to this problem, I would love to know as I just cannot find an approach I like.
n.b. I am literally nearing feature completion in Odin and I just want to tackle the final big problems
Edited by Ginger Bill
on
Reason: Typo
I kind of like the "vendoring" approach in Go but really dislike the $GOPATH approach.
Instead of having a global directory where packages are installed, each project should have dependencies install simply as local directories.
One possible source of complications is a convention used by a lot of people (might not apply to Odin, but who knows) where people put all the code inside a `src` directory. So if you just copy `lib_abc` directory, the code for the library won't be in `./lib_abc/module.odin`, but will be `./lib_abc/src/module.odin`, which would be really annoying because ideally I would just want to say `import "lib_abc/module.odin". I'm not sure what to do about that.
Another consideration to be taken into account, in my opinion, the meaning of import paths should not be affected by where you are right now. That's why having a 'global directory' for packages is not a good idea. I think having a line that says `import "module.odin"` should have exactly the same meaning whether you are the developer of `lib_abc` or a user of it who cloned it locally and is editing code inside the package. You'd think this is obvious but I had a lot of grief trying to do this in Go (editing the source code of a third-party package after installing it locally).
Instead of having a global directory where packages are installed, each project should have dependencies install simply as local directories.
One possible source of complications is a convention used by a lot of people (might not apply to Odin, but who knows) where people put all the code inside a `src` directory. So if you just copy `lib_abc` directory, the code for the library won't be in `./lib_abc/module.odin`, but will be `./lib_abc/src/module.odin`, which would be really annoying because ideally I would just want to say `import "lib_abc/module.odin". I'm not sure what to do about that.
Another consideration to be taken into account, in my opinion, the meaning of import paths should not be affected by where you are right now. That's why having a 'global directory' for packages is not a good idea. I think having a line that says `import "module.odin"` should have exactly the same meaning whether you are the developer of `lib_abc` or a user of it who cloned it locally and is editing code inside the package. You'd think this is obvious but I had a lot of grief trying to do this in Go (editing the source code of a third-party package after installing it locally).
I personally think the $GOPATH approach is fine and is that for your own projects, you are meant to set it so it installs locally rather than using a default directory. However, Odin already handles this sort of problem with the library collection system.
Since Go 1.6, Go will actually look for dependencies in a local "vendor" folder first. Plus, Go is finally getting some decent dependency management tools, so the $GOPATH situation is improving. But I digress.
I am also a fan of the Go approach, where a package is just a directory with source files in it. I think that aspect of their package design is very good, and any frustration I have with Go's packages is for other reasons. I think if you had the ability to absolutely or relatively reference a folder, you'd be fine. I guess you probably need multiple package search paths to make a standard library work, but that should be easy enough.
I'm inclined to think that the single-file library is an artifact of C and C++'s reliance on the preprocessor. I think passing directories around would be just fine.
(As a side note, using directories as packages can have other advantages, like the ability to include test code, documentation, benchmarks, etc. without affecting normal compilation.)
I am also a fan of the Go approach, where a package is just a directory with source files in it. I think that aspect of their package design is very good, and any frustration I have with Go's packages is for other reasons. I think if you had the ability to absolutely or relatively reference a folder, you'd be fine. I guess you probably need multiple package search paths to make a standard library work, but that should be easy enough.
I'm inclined to think that the single-file library is an artifact of C and C++'s reliance on the preprocessor. I think passing directories around would be just fine.
(As a side note, using directories as packages can have other advantages, like the ability to include test code, documentation, benchmarks, etc. without affecting normal compilation.)
Edited by Ben Visness
on
The problem with having a central place for shared libraries in Odin is the import "syntax" is different.
Say package A depends on package B and imports it using the "shared:" syntax, and there's also another package C that also depends on package B using the "shared:" but on a different and incompatible version, and I want to depend on both A and C.
ok, forget about this scenario with package C depending on a different and incompatible version of B. Just think about package A depending on package B being installed in the shared libraries folder. This means if I want to use package A in my project, I have no choice about how to install it. Sure, I can install package A itself locally, but then I'm _forced_ to install package B in the shared libraries folder.
The "shared" libraries folder is part of the external environment of the system and not a property of the project itself. Which means the project is not entirely self contained.
Controlling the location of the shared folder via environment variable would mitigate _this_ problem, but just in general I don't think it's a good idea because it's reminiscent of the "virtual environment" stuff from Python, which I think should not really exist.
The node.js solution of having all dependencies installed locally (in the local directory) is actually better because then the project is entirely contained within one directory. You can just copy the directory somewhere and it should work (at least in theory) offline.
1 2 3 | import "local/file.odin" // local directory import "core:file.odin" // system module import "shared:file.odin" // shared library |
Say package A depends on package B and imports it using the "shared:" syntax, and there's also another package C that also depends on package B using the "shared:" but on a different and incompatible version, and I want to depend on both A and C.
ok, forget about this scenario with package C depending on a different and incompatible version of B. Just think about package A depending on package B being installed in the shared libraries folder. This means if I want to use package A in my project, I have no choice about how to install it. Sure, I can install package A itself locally, but then I'm _forced_ to install package B in the shared libraries folder.
The "shared" libraries folder is part of the external environment of the system and not a property of the project itself. Which means the project is not entirely self contained.
Controlling the location of the shared folder via environment variable would mitigate _this_ problem, but just in general I don't think it's a good idea because it's reminiscent of the "virtual environment" stuff from Python, which I think should not really exist.
The node.js solution of having all dependencies installed locally (in the local directory) is actually better because then the project is entirely contained within one directory. You can just copy the directory somewhere and it should work (at least in theory) offline.
Edited by hasen.judy
on
Fair points about the difficulty of importing packages that depend on a shared library. Those problems kind of go away if you leave the package's location out of your import statements, and refer to packages by name only. That gives you the freedom to put libraries wherever they make sense, as long as they're part of the compiler's library search path.
I feel like that approach *should* be compatible with importing local packages as well. Depends how much you want to change the import syntax.
(I also am not really familiar with Odin's current handling of libraries so take my opinions with a grain of salt.)
I feel like that approach *should* be compatible with importing local packages as well. Depends how much you want to change the import syntax.
(I also am not really familiar with Odin's current handling of libraries so take my opinions with a grain of salt.)