Compare commits

...

6 Commits

12 changed files with 1621 additions and 133 deletions

327
README.md Normal file

@ -0,0 +1,327 @@
# diamond
### a golang ORM for mongodb that rocks 🎸~♬
# usage
## installation
run the following command in your terminal...
```shell
go get rockfic.com/orm
```
...and import the package at the top of your file(s) like so:
```go
package tutorial
import "rockfic.com/orm"
```
## Connect to the database
```go
package tutorial
import "rockfic.com/orm"
func main() {
orm.Connect("mongodb://127.0.0.1", "your_database")
}
```
this will create a connection and store it in the `DB` global variable. This global variable is used internally to
interact with the underlying database.
if you need to <sub><sub>~~why?~~</sub></sub>, you may also access the mongoDB client directly via the `orm.Client`
variable.
## Create a Model
to create a new model, you need to define a struct like this:
```go
package tutorial
import "rockfic.com/orm"
type User struct {
orm.Document `bson:",inline" coll:"collection"`
ID int64 `bson:"_id"`
Username string `bson:"username"`
Email string `bson:"email"`
Friends []User `bson:"friends"`
}
```
this on its own is useless. to actually do anything useful with it, you need to *register* this struct as a model:
```go
package tutorial
import "rockfic.com/orm"
func main() {
/* ~ snip ~ */
orm.ModelRegistry.Model(User{})
}
```
you can also pass multiple arguments to `orm.Model`, so long as they embed the `Document` struct.
you can access the newly created model like this:
```go
func main() {
/* ~ snip ~ */
userModel := orm.ModelRegistry.Get("User")
}
```
## Documents
any struct can be used as a document, so long as it embeds the `Document` struct. the `Document` struct is special, in
that it turns any structs which embed it into `IDocument` implementors.
`Document` should be embedded with a `bson:",inline" tag, otherwise you will end up with something like this in the
database:
```bson
{
"document": {
"createdAt": ISODate('2001-09-11T05:37:18.742Z'),
"updatedAt": ISODate('2001-09-11T05:37:18.742Z')
},
_id: 1,
username: "foobar",
email: "test@testi.ng",
friends: []
}
```
a `coll` or `collection` tag is also required, to assign the model to a mongodb collection. this tag is only valid on
the embedded `Document` field, and each document can only have one collection associated with it.
<sub>obviously.</sub>
<sub><sub>i mean seriously, who'd want to store one thing in two places?</sub></sub>
the recommended way to create a new document instance is via the `orm.Create` method:
```go
func main() {
/* ~ snip ~ */
user := orm.Create(User{}).(*User)
}
```
similarly, to create a slice of documents, call `orm.CreateSlice`:
```go
func main() {
/* ~ snip ~ */
users := orm.CreateSlice[User](User{})
}
```
lastly, let's implement the `HasID` interface on our document:
```go
func (u *User) GetId() any {
return u.ID
}
func (u *User) SetId(id any) {
u.ID = id.(int64)
}
```
#### but why do i need to implement this? :(
other ORMs rudely assume that you want your `_id`s to be mongodb ObjectIDs, which are **fucking ugly**, but also, more
importantly, *may not match your existing schema.*
by doing things this way, you can set your _id to be any of the following types:
- int
- int32
- int64
- uint
- uint32
- uint64
- string
- ObjectId
<sub>(if you hate yourself that much)</sub>
## finding/querying
`Find`, `FindOne`, `FindByID`, and `FindPaged` all return an `*orm.Query` object.
to get the underlying value, simply call `.Exec()`, passing to it a pointer to the variable in which you wish to store
the results.
### `Find` example
```go
func main() {
/* ~ snip ~ */
userModel := orm.ModelRegistry.Get("User")
if userModel != nil {
res := orm.CreateSlice(User{})
q, _ := userModel.Find(bson.M{"username": "foobar"})
q.Exec(&res)
}
}
```
## saving/updating
to save a document, simply call the `Save()` method. if the document doesn't exist, the ORM will create it, otherwise it
replaces the existing one.
## referencing other documents
it's possible to store references to other documents/structs, or slices to other documents. given the `User` struct we
defined above, which contains a slice (`Friends`) referencing other `User`s, we could change the `Friends` field in the
`User` type to be populateable, like so:
```diff
type User struct {
orm.Document `bson:",inline" coll:"collection"`
ID int64 `bson:"_id"`
Username string `bson:"username"`
Email string `bson:"email"`
- Friends []User `bson:"friends"`
+ Friends []User `bson:"friends" ref:"User"`
}
```
assuming we've filled a user's `Friends` with a few friends...
```go
func main() {
/* ~ snip ~ */
user := orm.Create(User{}).(*User)
for i := 0; i < 10; i++ {
friend := orm.Create(User{ID: int64(i + 2)}).(*User)
user.Friends = append(user.Friends, friend)
}
}
```
...in the database, `friends` will be stored like this:
```json5
{
// ~ snip ~ //
friends: [
2,
3,
4,
5,
6,
7,
8,
9,
10,
11
]
}
```
after retrieving a `User` from the database, you can load their `Friends` and their info by using the `Populate`function
on the returned `Query` pointer, like in the following example:
```go
func main() {
/* ~ snip ~ */
query, err := userModel.FindOne(bson.M{"_id": 1})
if err == nil {
result := orm.Create(User{}).(*User)
query.Populate("Friends").Exec(result)
fmt.Printf("%+v", result)
}
}
```
### Models vs Documents: what's the difference?
a **`Model`** is a static type that contains information about a given Document-like type. a **`Document`**, on the
other hand, is a full-fledged instances with its own set of methods.
if you come from an object-oriented programming background, you can envision the relationship between a `Model` and
`Document` something like this (using Java as an example):
```java
public class Document {
public void Pull(field String, elements ...Document) {
// ...
}
public void Append(field String, elements ...Document) {
// ...
}
public void Save() {
//...
}
public void Delete() {
//...
}
// model methods //
public static Document FindOne(HashMap<String, Object> query, MongoOptions options) {
// ...
}
public static ArrayList<Document> FindAll(HashMap<String, Object> query, MongoOptions options) {
// ...
}
}
```
## Struct Tags
the following tags are recognized by the ORM:
### `ref`
the syntax of the `ref` tag is:
```
`ref:"struct_name"`
```
where `struct_name` is the name of the struct you're referencing, as shown
in [Referencing other documents](#referencing-other-documents).
### `gridfs`
you can load files stored in [gridfs](https://www.mongodb.com/docs/manual/core/gridfs/) directly into your structs using
this tag, which has the following syntax:
```
`gridfs:"bucket_name,file_fmt"
```
where:
- `bucket_name` is the name of the gridfs bucket
- `file_fmt` is a valid [go template string](https://pkg.go.dev/text/template) that resolves to the unique file name.
all exported methods and fields in the surrounding struct can be referenced in this template.
currently, the supported field types for the `gridfs` tag are:
- `string`
- `[]byte`
### `idx`/`index`
indexes can be defined using this tag on the field you want to create the index for. the syntax is as follows:
`idx:{field || field1,field2,...},keyword1,keyword2,...`
supported keywords are `unique` and `sparse`.
`background` is technically allowed, but it's deprecated, and so will essentially be a no-op on mongodb 4.2+.
## acknowledgements
- [goonode/mogo](https://github.com/goonode/mogo), the project which largely inspired this one
# Further reading
see the [godocs]("https://git.tablet.sh/tablet/diamond-orm/src/branch/main/godoc.html") for more details :)

@ -172,12 +172,12 @@ func (d *Document) Pull(field string, a ...any) error {
if fv.Kind() != reflect.Slice {
return ErrNotASlice
}
outer:
for _, b := range a {
inner:
for i := 0; i < fv.Len(); i++ {
if reflect.DeepEqual(b, fv.Index(i).Interface()) {
fv.Set(pull(fv, i, fv.Index(i).Type()))
break outer
break inner
}
}
}

@ -39,21 +39,6 @@ func serializeIDs(input interface{}) interface{} {
return nil
}
}
/*var itagged interface{}
if reflect.ValueOf(itagged).Kind() != reflect.Pointer {
itagged = incrementTagged(&input)
} else {
itagged = incrementTagged(input)
}
taggedVal := reflect.ValueOf(reflect.ValueOf(itagged).Interface()).Elem()
if vp.Kind() == reflect.Ptr {
tmp := reflect.ValueOf(taggedVal.Interface())
if tmp.Kind() == reflect.Pointer {
vp.Elem().Set(tmp.Elem())
} else {
vp.Elem().Set(tmp)
}
}*/
switch vp.Elem().Kind() {
case reflect.Struct:
ret0 := bson.M{}
@ -72,10 +57,6 @@ func serializeIDs(input interface{}) interface{} {
bson.Unmarshal(marsh, &unmarsh)
for k, v := range unmarsh {
ret0[k] = v
/*if t, ok := v.(primitive.DateTime); ok {
ret0
} else {
}*/
}
} else {
_, terr := tag.Get("ref")
@ -92,8 +73,6 @@ func serializeIDs(input interface{}) interface{} {
rarr = append(rarr, getID(fv.Index(j).Interface()))
}
ret0[bbson.Name] = rarr
/*ret0[bbson.Name] = serializeIDs(fv.Interface())
break*/
} else if !ok {
panic(fmt.Sprintf("referenced model slice at '%s.%s' does not implement HasID", nameOf(input), ft.Name))
} else {

@ -10,6 +10,7 @@ var (
ErrOutOfBounds = errors.New("Index(es) out of bounds!")
ErrAppendMultipleDocuments = errors.New("Cannot append to multiple documents!")
ErrNotSliceOrStruct = errors.New("Current object or field is not a slice nor a struct!")
ErrUnsupportedID = errors.New("Unknown or unsupported id type provided")
)
const (

1229
godoc.html Normal file

File diff suppressed because one or more lines are too long

@ -32,12 +32,12 @@ func parseFmt(format string, value any) string {
return w.String()
}
func bucket(gfsRef GridFSReference) *gridfs.Bucket {
func bucket(gfsRef gridFSReference) *gridfs.Bucket {
b, _ := gridfs.NewBucket(DB, options.GridFSBucket().SetName(gfsRef.BucketName))
return b
}
func gridFsLoad(val any, g GridFSReference, field string) any {
func gridFsLoad(val any, g gridFSReference, field string) any {
doc := reflect.ValueOf(val)
rdoc := reflect.ValueOf(val)
if doc.Kind() != reflect.Pointer {
@ -125,8 +125,8 @@ func gridFsSave(val any, imodel Model) error {
}
_, err := structtag.Parse(string(ft.Tag))
panik(err)
var gfsRef *GridFSReference
for kk, vv := range imodel.GridFSReferences {
var gfsRef *gridFSReference
for kk, vv := range imodel.gridFSReferences {
if strings.HasPrefix(kk, ft.Name) {
gfsRef = &vv
break

@ -17,7 +17,7 @@ type Counter struct {
}
func getLastInColl(cname string, id interface{}) interface{} {
var opts *options.FindOneOptions = options.FindOne()
var opts = options.FindOne()
switch id.(type) {
case int, int64, int32, uint, uint32, uint64, primitive.ObjectID:
@ -25,7 +25,7 @@ func getLastInColl(cname string, id interface{}) interface{} {
case string:
opts.SetSort(bson.M{"createdAt": -1})
default:
panic("unsupported id type provided")
panic(ErrUnsupportedID)
}
var cnt Counter

@ -10,15 +10,16 @@ import (
"unsafe"
)
// Model - "base" struct for all queryable models
// Model - type which contains "static" methods like
// Find, FindOne, etc.
type Model struct {
typeName string `bson:"-"`
Idx int
Type reflect.Type
Collection string
References map[string]Reference
Indexes map[string][]InternalIndex
GridFSReferences map[string]GridFSReference
Type reflect.Type
collection string
gridFSReferences map[string]gridFSReference
idx int
references map[string]Reference
typeName string `bson:"-"`
}
// HasID is a simple interface that you must implement
@ -59,9 +60,9 @@ func (m *Model) setTypeName(str string) {
func (m *Model) getColl() *mongo.Collection {
_, ri, ok := ModelRegistry.HasByName(m.typeName)
if !ok {
panic(fmt.Sprintf("the model '%s' has not been registered", m.typeName))
panic(fmt.Sprintf(errFmtModelNotRegistered, m.typeName))
}
return DB.Collection(ri.Collection)
return DB.Collection(ri.collection)
}
func (m *Model) getIdxs() []*mongo.IndexModel {
@ -80,7 +81,7 @@ func (m *Model) getIdxs() []*mongo.IndexModel {
func (m *Model) getParsedIdxs() map[string][]InternalIndex {
_, ri, ok := ModelRegistry.HasByName(m.typeName)
if !ok {
panic(fmt.Sprintf("model '%s' not registered", m.typeName))
panic(fmt.Sprintf(errFmtModelNotRegistered, m.typeName))
}
return ri.Indexes
}
@ -154,7 +155,9 @@ func (m *Model) FindOne(query interface{}, options ...*options.FindOneOptions) (
rip := coll.FindOne(context.TODO(), query, options...)
raw := bson.M{}
err := rip.Decode(&raw)
panik(err)
if err != nil {
return nil, err
}
qqn := ModelRegistry.new_(m.typeName)
idoc, ok := qqn.(IDocument)
if ok {
@ -222,6 +225,9 @@ func Create(d any) any {
return what
}
// CreateSlice - convenience method which creates a new slice
// of type *T (where T is a type which embeds Document) and
// returns it
func CreateSlice[T any](d T) []*T {
r, _, _ := createBase(d)
rtype := r.Type()
@ -230,7 +236,3 @@ func CreateSlice[T any](d T) []*T {
newItem.Elem().Set(reflect.MakeSlice(rslice, 0, 0))
return newItem.Elem().Interface().([]*T)
}
func (m *Model) PrintMe() {
fmt.Printf("My name is %s !\n", nameOf(m))
}

@ -199,9 +199,9 @@ func (q *Query) LoadFile(fields ...string) *Query {
_, cm, _ := ModelRegistry.HasByName(q.model.typeName)
if cm != nil {
for _, field := range fields {
var r GridFSReference
var r gridFSReference
hasAnnotated := false
for k2, v := range cm.GridFSReferences {
for k2, v := range cm.gridFSReferences {
if strings.HasPrefix(k2, field) {
r = v
hasAnnotated = true
@ -227,14 +227,14 @@ func (q *Query) Populate(fields ...string) *Query {
var r Reference
for k2, v := range cm.References {
for k2, v := range cm.references {
if strings.HasPrefix(k2, field) {
r = v
break
}
}
if r.Exists {
if r.exists {
// get self
// get ptr
// find
@ -263,7 +263,7 @@ func (q *Query) Populate(fields ...string) *Query {
ref = ref.Elem()
}
src := ref.Index(i).Interface()
inter := populate(r, refColl.Collection, val2, field, src)
inter := populate(r, refColl.collection, val2, field, src)
if reflect.ValueOf(inter).Kind() == reflect.Pointer {
slic.Elem().Set(reflect.Append(slic.Elem(), reflect.ValueOf(inter)))
} else {
@ -272,7 +272,7 @@ func (q *Query) Populate(fields ...string) *Query {
}
tmp1 = slic.Interface()
} else {
tmp1 = populate(r, refColl.Collection, rawDoc, field, reflect.ValueOf(q.doc).Interface())
tmp1 = populate(r, refColl.collection, rawDoc, field, reflect.ValueOf(q.doc).Interface())
}
q.doc = tmp1
}
@ -290,21 +290,11 @@ func (q *Query) reOrganize() {
slic := reflect.New(reflect.SliceOf(typ))
for _, v2 := range arr {
inter := reflect.ValueOf(rerere(v2, typ))
/*if inter.Kind() == reflect.Pointer {
inter = inter.Elem()
}*/
slic.Elem().Set(reflect.Append(slic.Elem(), inter))
}
trvo = slic.Elem()
} else {
trvo = reflect.ValueOf(rerere(q.rawDoc, reflect.TypeOf(q.doc)))
/*for {
if trvo.Kind() == reflect.Pointer {
trvo = trvo.Elem()
} else {
break
}
}*/
}
resV := reflect.ValueOf(q.doc)

@ -5,7 +5,6 @@ import (
"fmt"
"log"
"reflect"
"strings"
"sync"
"github.com/fatih/structtag"
@ -30,10 +29,10 @@ type Reference struct {
// field kind (struct, slice, ...)
Kind reflect.Kind
Exists bool
exists bool
}
type GridFSReference struct {
type gridFSReference struct {
BucketName string
FilenameFmt string
LoadType reflect.Type
@ -45,7 +44,7 @@ type TModelRegistry map[string]*Model
// ModelRegistry - the ModelRegistry stores a map containing
// pointers to Model instances, keyed by an associated
// model name
var ModelRegistry = make(TModelRegistry, 0)
var ModelRegistry = make(TModelRegistry)
// DB - The mongodb database handle
var DB *mongo.Database
@ -59,27 +58,7 @@ var NextStringID func() string
var mutex sync.Mutex
func getRawTypeFromTag(tagOpt string, slice bool) reflect.Type {
var t reflect.Type
switch strings.ToLower(tagOpt) {
case "int":
var v int64 = 0
t = reflect.TypeOf(v)
case "uint":
var v uint = 0
t = reflect.TypeOf(v)
case "string":
var v = "0"
t = reflect.TypeOf(v)
}
if slice {
return reflect.SliceOf(t)
}
return t
}
func makeGfsRef(tag *structtag.Tag, idx int) GridFSReference {
func makeGfsRef(tag *structtag.Tag, idx int) gridFSReference {
opts := tag.Options
var ffmt string
if len(opts) < 1 {
@ -101,7 +80,7 @@ func makeGfsRef(tag *structtag.Tag, idx int) GridFSReference {
}
}
return GridFSReference{
return gridFSReference{
FilenameFmt: ffmt,
BucketName: tag.Name,
LoadType: typ,
@ -117,7 +96,7 @@ func makeRef(idx int, modelName string, fieldName string, ht reflect.Type) Refer
Model: modelName,
HydratedType: ht,
Kind: ht.Kind(),
Exists: true,
exists: true,
FieldName: fieldName,
}
}
@ -127,17 +106,17 @@ func makeRef(idx int, modelName string, fieldName string, ht reflect.Type) Refer
FieldName: fieldName,
HydratedType: ht,
Kind: ht.Kind(),
Exists: true,
exists: true,
}
}
panic("model name was empty")
}
func parseTags(t reflect.Type, v reflect.Value) (map[string][]InternalIndex, map[string]Reference, map[string]GridFSReference, string) {
func parseTags(t reflect.Type, v reflect.Value) (map[string][]InternalIndex, map[string]Reference, map[string]gridFSReference, string) {
coll := ""
refs := make(map[string]Reference)
idcs := make(map[string][]InternalIndex)
gfsRefs := make(map[string]GridFSReference)
gfsRefs := make(map[string]gridFSReference)
for i := 0; i < v.NumField(); i++ {
sft := t.Field(i)
@ -183,10 +162,6 @@ func parseTags(t reflect.Type, v reflect.Value) (map[string][]InternalIndex, map
break
}
if refTag, ok := tags.Get("ref"); ok == nil {
// ref:"ModelName,refType"
/* if len(refTag.Options) < 1 {
panic("no raw type provided for ref")
} */
sname := sft.Name + "@" + refTag.Name
refs[sname] = makeRef(i, refTag.Name, sft.Name, sft.Type)
}
@ -239,7 +214,7 @@ func (r TModelRegistry) HasByName(n string) (string, *Model, bool) {
// Index returns the index at which the Document struct is embedded
func (r TModelRegistry) Index(n string) int {
if v, ok := ModelRegistry[n]; ok {
return v.Idx
return v.idx
}
return -1
}
@ -247,7 +222,7 @@ func (r TModelRegistry) Index(n string) int {
func (r TModelRegistry) new_(n string) interface{} {
if name, m, ok := ModelRegistry.HasByName(n); ok {
v := reflect.New(m.Type)
df := v.Elem().Field(m.Idx)
df := v.Elem().Field(m.idx)
do := reflect.New(df.Type())
d := do.Interface().(IDocument)
//d := df.Interface().(IDocument)
@ -259,6 +234,14 @@ func (r TModelRegistry) new_(n string) interface{} {
return nil
}
func (r TModelRegistry) Get(name string) *Model {
model, ok := r[name]
if !ok {
return nil
}
return model
}
// Model registers models in the ModelRegistry, where
// they can be accessed via a model's struct name
func (r TModelRegistry) Model(mdl ...any) {
@ -305,29 +288,29 @@ func (r TModelRegistry) Model(mdl ...any) {
}
inds, refs, gfs, coll := parseTags(t, v)
if coll == "" {
panic(fmt.Sprintf("a model needs to be given a collection name! (passed type: %s)", n))
panic(fmt.Sprintf("a Document needs to be given a collection name! (passed type: %s)", n))
}
ModelRegistry[n] = &Model{
Idx: idx,
idx: idx,
Type: t,
Collection: coll,
collection: coll,
Indexes: inds,
References: refs,
GridFSReferences: gfs,
references: refs,
gridFSReferences: gfs,
}
}
for k, v := range ModelRegistry {
for k2, v2 := range v.References {
if !v2.Exists {
for k2, v2 := range v.references {
if !v2.exists {
if _, ok := ModelRegistry[v2.FieldName]; ok {
tmp := ModelRegistry[k].References[k2]
ModelRegistry[k].References[k2] = Reference{
tmp := ModelRegistry[k].references[k2]
ModelRegistry[k].references[k2] = Reference{
Model: k,
Idx: tmp.Idx,
FieldName: tmp.FieldName,
Kind: tmp.Kind,
HydratedType: tmp.HydratedType,
Exists: true,
exists: true,
}
}
}

@ -136,7 +136,7 @@ func genChaps(single bool) []chapter {
ret = append(ret, chapter{
ID: primitive.NewObjectID(),
Title: fmt.Sprintf("-%d-", i+1),
Index: int(i + 1),
Index: i + 1,
Words: 50,
Notes: "notenotenote !!!",
Genre: []string{"Slash"},
@ -202,10 +202,6 @@ func initTest() {
author.ID = 696969
ModelRegistry.Model(band{}, user{}, story{})
}
func after() {
err := DBClient.Disconnect(context.TODO())
panik(err)
}
var metallica = band{
ID: 1,

25
util.go

@ -64,7 +64,7 @@ func getNested(field string, aValue reflect.Value) (*reflect.Type, *reflect.Valu
}
aft := value.Type()
dots := strings.Split(field, ".")
if value.Kind() != reflect.Struct /*&& arrRegex.FindString(dots[0]) == ""*/ {
if value.Kind() != reflect.Struct {
if value.Kind() == reflect.Slice {
st := reflect.MakeSlice(value.Type().Elem(), 0, 0)
for i := 0; i < value.Len(); i++ {
@ -85,8 +85,6 @@ func getNested(field string, aValue reflect.Value) (*reflect.Type, *reflect.Valu
} else {
return &aft, &value, nil
}
/*ft := value.Type()
*/
}
ref := value
if ref.Kind() == reflect.Pointer {
@ -96,7 +94,7 @@ func getNested(field string, aValue reflect.Value) (*reflect.Type, *reflect.Valu
if arrRegex.FindString(dots[0]) != "" && fv.Kind() == reflect.Slice {
matches := arrRegex.FindStringSubmatch(dots[0])
ridx, _ := strconv.Atoi(matches[0])
idx := int(ridx)
idx := ridx
fv = fv.Index(idx)
}
@ -135,7 +133,7 @@ func incrementInterface(t interface{}) interface{} {
case primitive.ObjectID:
t = primitive.NewObjectID()
default:
panic("unknown or unsupported id type")
panic(ErrUnsupportedID)
}
return t
}
@ -194,20 +192,3 @@ func normalizeSliceToDocumentSlice(in any) *DocumentSlice {
}
return &ret
}
/*func normalizeDocSlice(iput reflect.Value) reflect.Value {
idslice, idsliceOk := iput.Interface().(IDocumentSlice)
dslice, dsliceOk := resV.Interface().(DocumentSlice)
switch {
case idsliceOk:
case dsliceOk:
default:
return iput
}
//if {
// resV.Set(trvo.Elem())
//} else {
//
//}
}*/