How do I delete a record by its primary key without first doing a SELECT statement to load the entity?
You can use dummy object:
var entity = new YourEntity { Key = yourKey };
context.Entities.Attach(entity);
context.Entities.DeleteObject(entity);
context.SaveChanges();