Google App Engine Assigning Entity Groups / Parent Keys, Unique constraint

Question

I have a Kind of 'Customer'. I want to run a transaction that locks the entire Kind when a new 'Customer' is about to be inserted. The transaction would first query to check that the new 'Customer' Name does not already exist, then the 2nd part of the transaction runs the insert if no matches are found. This way I'm enforcing a Unique Constraint (and also restricting the operation to approx 1 insert per second).

My unsatisfactory solution to getting all my 'Customer' entitys in the same entity group is to create a Kind called 'EntityGroups', with a single record called 'CustomersGroup'. This one record is used every time as the Parent of newly created 'Customer' entities, thereby grouping the entire Kind into one entity group.

My question is: I am concerned about using a phantom record such as 'CustomerGroup' because if anything happened and it were lost or deleted, I could not assign any new 'Customer' entities to the same group! I imagine it would be better to assign the Parent of each 'Customer' entity a static arbitrary parent, such as '1111111'? I think the terminology is "virtual root entity", how do I do this?

Please help with any advice on how I can best handle this!

score 3 · Answer 1 · answered Oct 30 '12 at 16:30

3

Why don't you use: NDB's get_or_insert: Transactionally retrieves an existing entity or creates a new one.

https://developers.google.com/appengine/docs/python/ndb/modelclass#Model_get_or_insert

answered Oct 30 '12 at 16:30

voscausa

11,253
2
39
67

Thanks Voscausa. Let say I have a Customer called ACME. I query for him and there is no match. I begin a put operation for ACME, just as another user begins putting their ACME entry into the db. We end up with 2 ACME's in the datastore, when business logic dictates no duplicates are permitted. I cant store ACME as the KEY_NAME (where get_or_insert would be the solution, and uniqueness is guaranteed) because business logic dictates the company name field might change over time! – GrantsV Oct 30 '12 at 17:09
Assign the company a unique ID then and use that as it's ID in the datastore. That will never change, even if the company name does. Then it's just one get_by_id to get the current company name. Using get_or_insert is exactly the thing do to here as the answer suggests! Do it! :) – Paul Collingwood Oct 30 '12 at 17:29

score 3 · Answer 2 · answered Oct 30 '12 at 18:31

Your CustomerGroup record does not need to exist for it to act as a parent. Just create it's key by hand and assign it as the parent to the record in question.

You don't need to worry about it being deleted if it does not exist!

When you create a model and set another as it's parent the system does not check (nor does it need to ) that that model actually exists at all.

So for example:

rev_key = ndb.Key('CustomerGroup', '11111', 'Customer', 'New_Customer_Name')

Yet a model with a key of: ('CustomerGroup', '11111') does not actually exist but it can still be in the ancestor chain.

Awesome Paul! Thats just what I was looking for! – GrantsV Oct 30 '12 at 20:28 — GrantsV, Oct 30 '12 at 20:28

score 1 · Answer 3 · answered Oct 30 '12 at 20:27

GrantsV, you can achieve this by creating a proxy entity for each unique constraint and using cross-group transactions to commit the constraints with the normal writes.

class UniqueConstraint(db.Model):
  # Consider adding a reference to the owner of the constraint.
  @db.transactional(propagation=db.MANDATORY, xg=True)
  @classmethod
  def reserve(cls, kind, property, value):
    key = cls.__get_key(kind, property, value)
    if db.get(key):
      raise Exception  # Already exists
    cls(key=key).put()

  @db.transactional(propagation=db.MANDATORY, xg=True)
  @classmethod
  def release(cls, kind, property, value):
    db.delete(cls.__get_key(kind, property, value))

  @classmethod
  def __get_key(cls, kind, property, value):
    # Consider using a larger entity group.
    return db.Key.from_path(cls.kind(), '%s:%s:%s' % (kind, property, value))
    # To restrict to 1 insert per second per kind, use:
    # return db.Key.from_path(cls.kind(), kind, cls.kind(), '%s:%s' % (property, value))

When using the `@db.transactional` decorator together with `@classmethod`, the `@classmethod` line has to be placed above the `@db.transactional`. Otherwise this code will raise an exception when invoked. — typeracer, Jun 09 '20 at 21:15

score 0 · Answer 4 · answered Oct 30 '12 at 15:44

0

You can create a parent entity, like this:

class CustomerParent(ndb.Model):
    pass

Then you instantiate and store your parent entity:

customers_parent = CustomerParent()
customers_parent.put()

Finally, when you create all your customer entities, you specify the parent:

a_customer = Customer(parent=customers_parent.key, ...)
a_customer.put()

Hope this helps!

answered Oct 30 '12 at 15:44

Thanos Makris

3,115
2
17
26

You don't want to go the route of using a single parent. That limits your performance significantly. – dragonx Oct 30 '12 at 16:50
Thanks Thanos, this is exactly what I am doing. But my concern is what if my master "CustomerParent" Entity gets deleted, or another entry is accidently created. I then cant add new Customers to the original entity group or new customers are added to a new entity group! – GrantsV Oct 30 '12 at 16:54
Thanks Dragonx, but this performance limitation isnt a problem. What is a show stopper is creating 2 entities with the same Name field. And I cant use the KEY_NAME to enforce uniquness because the Name field may change in time. – GrantsV Oct 30 '12 at 16:56
You do know that the master "CustomerParent" entity does not actually have to exist but you can still create models that have it as it's parent, right? – Paul Collingwood Oct 30 '12 at 17:31

Google App Engine Assigning Entity Groups / Parent Keys, Unique constraint

4 Answers4