Might be related
Big step forward in Modular Database Applications with DataObjects
Published 2010-09-01 00:00:00
Being a Software Developer is all about developing applications faster, and delivering quicker. At the same time ensuring that quality is not lost and readibility is kept. DB_DataObjects is one of the key tools in my productivity toolkit. It was originally designed as a way of ensuring that shared database related code ends up in the right place, In the Model layer of the application, rather than the View or Controller (or worse mixed in to some hybrid PHP HTML garbage..), along with doing cool stuff like Query building etc.
Over the years, of using DataObjects, I've built up quite a library of reusable DataObjects, which to some degree can be plugged in to any project. A Person object that handles login/authentication and authorization (working with the Permissions and Group objects). A Image object that handles storage of Images, and Files that can provide File type conversion for rendering (using things like unoconv, etc.)
HTML_FlexyFramework Provides the basic 'Controller', It's one of those very lightweight, get out of your way, 2 small classes bit of code. In some sense's it's not really a Framework (well 2 classes could hardly classify as that). More a generic Controller, and a Base class for a Page, just implementing the bare bones stub features, like get/post,getAuth() and implementing the output method to run HTML_Template_Flexy.
What I've built ontop of that is the Pman Components, originaly based on a project codenamed Pman (for project management). Which ended up being the most re-usable code I've developed to date. It's the foundation for around 6 completely different projects now, sharing a large amount of modular code. The basic code of the Pman Project is Pman.Base and Pman.Core, in themselves they do not provide much more than the basic Interface outline.
The directory structure for projects
- /index.php (the bootstrapper and installed configuration- the link should show a rather messy example)
- Pman.Base (git submodule)
- Pman.Core (git submodule)
- Pman.Admin (git submodule)
- ......... (any other submodules for the project)
- 'Pman.Base' files and where they end up in the repositor.
- Pman.php (alias of Pman.Base/Pman.php) - this provides base feature for any custom code, like jerr(), jdata() and jok() for sending database via JSON in standard formats.
- Pman - Technically where all the real files for the submodules go..
- Pman/I18N.php (alias of to Pman.Base/Pman/I18N.php) - To be phased out eventually - most of the code is now in Pman.Core/I18N.php
- Pman/Login.php (alias of to Pman.Base/Pman/Login.php) -The base provider of Login Queries - both initial login, background information refresh and logout
- Pman/Images.php (alias of to Pman.Base/Pman/Images.php) - Generic provider code to deliver images to the interface (either as attachments, thumbnails or the real thing.)
- Pman/templates/images (alias of to Pman.Base/Pman/templates/images) - contains images and icons.. that will probably be moved upstream into Roo.
- 'Pman.Core' files aliased into the 'Pman/Core/'
- Pman.js = The Previous base class for Pman. Some of this code has been moved to Roo.XComponents, but it still provides a few usefull methods for all applicaitons, like Pman.download() etc..
- Pman.Dialog.*.js - some standard dialogs for editing generic data.
- DataObjects - the Core DB_DataObject defintionts.
- template - the standard master template
- Project Specific code
A Changing Data Model
The background here is that, as I re-used more and more of these componets, there would frequently be an issue where a particular project needed the modification of a core database component. In our example it was the adding of 'dispatch port' to the Company Table. This was a specific requirement for one project, which could have been impemented using a Companys_Extra table or some kind of 'properties' hack (which is what Midgard used to do back in early 2005). However the best solution is really to modify the base Table, so that it can be searched/ordered etc, and in the long run I'm looking at having a 'Designer Module' similar to SugarCRM / netsuite, that allows the end users to actually modify the database structure, adding colums as they need.
The Knock-on effect however for this small change is, unfortunatly it often breaks other code depending on the compoents, due to the process that is involved.
a) Modify the database changes file. normally in the DataObjects directory of the Component
b) Run it against the Database, updating the Data Definition for tables in that module.
b) Run the generator on the module, so that the schema.ini file is regenerated.
This works fine on the development server, and usually fine on the Live server for that project, as you remember to run the database changes from the relivant Module when you do the next install, Where this falls apart is usually a month later...
You go back to working on a project that was on hold for a while, and start by updating the modules in the project. Suddenly Authentication fails, or no data is available in the interface... - All this is due to the fact that the Database Queries being built expect certain columns to be there (based on the schema.ini file) and since you have forgotten to update the database all the queries break.. Not usually a big issue as the builder module has some code in it to apply all the database changes code, by runing the bootstrapper in CLI mode. - But still it's annoying.
The other issue is that you really do not need the Company Table to contain a 'dispatch port' when no other project is going to use it.
I have been thinking about the solution to this for some time, however, just before I jumped into refreshing some code in a project, I thought this would be the ideal time to solve this problem, and speed up my development process in the long run.
The change involved
a) HTML_FlexyFramework changes
dataObjectsCache = true (defaults to ON)
dataObjectsCacheExpires = 3600 (database Cache file expires every hour..)
which can be called to force regeneration of the ini files (stored in the session directory)
HTML_FlexyFramework_Generator - a tiny wrapper around DB_DataObject_Generator, that is loaded and called only when regeneration takes place.
- when you load the main page (not when you do JSON calls) it calls generateDataObjectCache(), which will generate the schema files if they have expired. - if you are running with Pman[isDev] then it will run them every time (eg. force generation)..
- new get Request argument _requestMeta
which adds to the data return value the JsonReader meta arguments which include schema type information, so you do not need the schema information in the UI (reducing the size of the UI considerably). note schema data is only returned if rows of data where also returned..
- new property metaFromRemote - a flag to indicate that the meta data was recieved from a remote source (and hence probably does not need fetching again).
- load method modified to check for metaFromRemote, if it is set to false, then _requestMeta is added to the data Query. (hence schema is only queried once).
All this should mean that I can move application specific database changes to core tables into their respective modules, and out of core..
- FIFO on xtuple
- PHP just does some things better. cloud backups, pecl-expect
- Roo J Solutions Growing into 2013, always recruiting, and developing differently
- Interesting Problems.
- Prettybooked.com turning a startup into a reality.
- Migrating off Netsuite - The hidden cost of Clouds..
- Roo.XComponent introduction