Project Peach
We need a modern browser to run our WebApp. Because you have either a legacy browser or have javascript disabled this is the best we can do!
2011: Back to the Future
07/04/2017
Introductions: Who are we?
John Ince
  • Been in software engineering professionally since 1987, writing code!
  • Consultant at Yorkshire Water, Unilever, Rapiscan Systems, Senior Engineer at IBM and several Startups
  • Diverse commercial projects, schematics, b2b connectivity, cognos business intelligence, RadNuke detectors
  • Skillset C, C++, win APIs for coding on windows & linux/unix. GUI and backend server work.
  • Currently framework architect, c++ server framework coding, sales.
  • Darren Ince
  • Professional software developer since 2005
  • Software engineer at AutoCab, consulant at Amey, Rapiscan Systems and several Startups
  • Diverse commercial projects, autocab, denials management, amey management, RadNuke R&D
  • Skillset C, C++, JS, HTML5, CSS3, SASS, web development stack, Full Stack
  • Currently our graphic artist, server and browser framework developer
  • Been working together as a team, on and off, since 2003. Our current startup since late 2011. Project Peach born. Most of what follows is an edited version of our programming mission statement from back in 2011 and where we were at back in 2014. It formed the basis of a presentation of live webapps for the UK Met Office in that year. Read on...
    Project Peach: What?
  • A stealth startup owned and funded entirely by ourselves with no outside investment. >£400k so far.
  • Five years developing our framework to accomplish the aims defined below
  • At the point of becoming a commercial company with our first web applications delivered
  • Looking to spread the word about our modern realtime framework and modern development processes
  • Project Peach: Aims
  • Investigate and use the latest technologies to create a more modern development process. Help us all build better apps!
  • Use html5 standards/specifications as a starting point to design and build our framework
  • Accept that the modern web browser is not simply a web browser, it's a fully fledged frontend GUI development studio
  • Create a client framework, server framework and the messaging connectivity between
  • Create live, realtime, web applications which are more like applications then websites. Use push messaging NOT polling
  • Use 'open source' software for all our library requirements. Make our framework 'open source' to our clients not free software
  • Not to be tied to any proprietary technology or companies technology! MS .NET example. Always have the source code
  • Not be tied to any particular OS for either client or server framework runtime requirements
  • Not to be tied to any particular browser, just a modern browser! Limited backwards support is acceptable!
  • To be scale our application GUI gracefully across all devices, mobile, tablet, laptop with one source code base
  • Remove the need to build apps for specific devices, one application fits all
  • Clear demarcation between markup, presentation/visual formatting and code. Graphic designer vs coder
  • Work as well with a finger as with a mouse pointer
  • To scale well as an application, run locally, run Intranet or Internet
  • Scale well using infrastructure upgrading, suitable for server farming
  • High performance messaging server, multi-threaded, exception safe and software scalable
  • Additions and improvements to our framework are available to all framework products and visa versa
  • To speed up development time by providing coding libraries and out of the box functionality
  • Proof. Can we migrate any applications we have previously written into our framework? YES
  • Modern Development
    Client Side
  • Modern Browsers for UI/UX
  • Chrome, safari, opera, firefox, IE11 plus mobile versions. All self updating browsers. All implementing html5 specifications
  • What's a browser for? Simply feeding from a web server or more? Request files, html, images and render document. historically YES
  • With html5 websockets there is a permanent browser/server connection to send and receive messages independant of the Web server!
  • An IDE for free! A visual designer, Javascript editor, interactive debugger, profiling tools, DOM tools, etc
  • Access to hardware like webcam, location services GPS, etc. In browser programmable access to hardware resources
  • Software implementations of client side html5 specifications like webrtc, websockets, notifications, localstorage, etc
  • Design UI/UX in a visual interactive way using html and css
  • A full programming environment using Javascript (we emulate OO) OR alternative languages like DART (more OO/class based)
  • Web components: shadow DOM, html templates, reusable widgets from html/css/js. Example html form controls all widgets
  • SCSS: sassy stylesheets adding variables, inheritance, functions, and parameterisation to CSS. Chrome dev tools native support
  • Server Side
  • Web Servers and Websockets for applications
  • Web servers are great for file serving. Connect, Get, Disconnect. We can use this to host static files like images and pages
  • Web servers can run code. PHP, .NET, ISAPI, .... Same model of connect and disconnect but useful for templated pages
  • Web server connections can be upgraded to web socket connections via a standard and a handshake. Permanent connection
  • Websocket servers are simple socket servers. listen, accept, connect. Tried and tested technology
  • A permanent connection to the server provides a scalable application messaging opportunity
  • A websocket server can preserve state natively for the connected browser
  • Browsers to server connections form a network of indirectly connected computers for messaging opportunities
  • A web socket server is a native application. We can build shared data models, access databases, map messages to events
  • Make a distinction between the websocket application and the application specifics to run multiple instances
  • Reuse server side business logic code and create stunning new browser application interfaces
  • What can we build today?
  • A traditional hyperlinked website that simply uses a web server and the transcient http connection
  • A traditional website using some behind the scenes web access XMLHTTPRequest/AJAX to avoid server roundtrips for data - polling
  • A hybrid website/webapp mix. Mostly website with additional 'push' live features via websockets
  • A webapp more like application than website, code and data models on server and client
  • A framework webapp with optional built in features out of the box, skype, IM, private social media, live controls, live push data
  • We can take a traditional windows application, modernise the UI in browser and reuse the business logic code on the server
  • Host the processing where it makes sense on the server or locally in the client browser, native code or interpreted JS code
  • Render a visualisation of an object model in a browser whose dynamic data model is server hosted, push only updates - live
  • Create indirect and direct connections of browser to server to browser. Inter-browser messaging and direct browser messaging
  • A webapp can work on a single machine, local networks or over the Internet
  • Live data forms with multiuser concurrent editing with realtime push updating
  • Traditional database application via our abstracted DB access classes. Currently sqllite support. MySQL just needs implementation
  • Limited only by our imagination...
  • What's your framework then?
    Messaging system core. Browser to server to browser realtime permanent connection
    A server component
  • A core messaging websockets server connecting browsers to server. A high performance implementation of websocket server spec.
  • A mechanism to map messages to fire events, run code handlers on the server
  • A map of connected browsers and a routing mechanism to pass messages to other browsers. Pass through
  • Generate outbound messages in response to server events and pass to subscribed browsers
  • Asynchronous Receive message, inbound message instance, thread instance, run handler, generate outbound message(s), send msg(s)
  • Inbuilt security protection from cross domain attacks and secondary protocol handshake. Only valid domain browser connections
  • A client/browser component
  • Intelligent persistent connection from browser to server, reconnecting on error
  • A client messaging library to send messages and receive responses to/from the server component
  • A Javascript event handling mechanism that fires JS handler code in response to messages
  • Page subscription to messages of interest to the page. Server only sends to registered observers
  • Send message, continue code, inbound response message event fired, handler code run
  • Inbound and outbound messages are independent of one another. Event model
  • Admin panel management
  • Browser hosted. Manage from mobile, tablet or desktop
  • Manages connections and statistics. Who's connected in realtime down to IP address
  • Thread pool management and statistics
  • Message performance for send and receive messaging and queuing tasks
  • Visualisation of the low level messaging server
  • Alert system to notify admins of issues
  • Live, realtime true push updating
    Real time browser visualisation of server object model
  • A connected browser page can request interest to a server hosted data object model
  • Initial client visualisation/rendering of the model during page intialisation via messaging. full model
  • User remains on page. watching. No long polls or timed polls causing network traffic
  • Event occurs that changes the server data model, either another user action or streaming data API
  • All subsribers to the data model, all connected browsers who are on the page are 'pushed' the update - only the change
  • Browser page update handler event is fired, handler code updates DOM, browser page updates
  • Interrupt driven, no unnecessary network traffic, minimum data transfer, realtime updating live
  • Fast streaming data APIs that update server object models animate browser visualisations. Only dirty, changed data, over network
  • Live forms and edit controls
  • Server hosted live controls/forms enable controlled safe concurrent multi-user data editing
  • Scenario: 3 admin users looking at the same user record in a form. Data is only as up to date as page initialisation
  • Each form instance registers with the server a number of 'live edit controls', for example first, middle and last name
  • Admin #1 starts to edit the middle name. What about Admin #2 and #3? Their data is now out of date
  • In live edits our admin #2 and #3 see middle name go read only, live changes are shown from user #1 until leaves control
  • Admin #2 edits first name concurrently with admin #1's middle name changes
  • Admin #3 sees both changes occuring live and who's changing them
  • Perfect harmony with multi-user concurrency. No changes are lost on save due to stale page initialisation values
  • Human to human Connectivity
    User and Guest login
  • A optional user login model with simple signup and guest support
  • Treat guests as near equals. Build confidence. Make signup simple, just username, avatar, captcha. Good conversion
  • Private social media open to guests, restricted. VoIP, IM is for users only because of friendship networks
  • Full administrator management of users. Optionally admin invitation only for user management
  • Friendship networks
  • Build a friendship network to tailor private social media from public timeline. Create workgroups
  • Users can search, add, remove friends just like facebook/twitter models
  • Friendship network can be automatic, public friendships
  • In browser Skype
  • webRTC standard to VoIP from browser to browser, user to user, friend to friend. Guests restricted to admin users
  • Voice or video direct between browsers without indirection through our messaging server (STUN/TURN server support)
  • Support for this technology built into the browser however handshake support is not! Need to handshake browsers before connect
  • Could be used effectively on internal/external networks to replace traditional landline calls
  • Use with great effect in contact or support pages. A more modern alternative to form, textedit, submit pages
  • rtcDataConnection available to transfer data in the same way as VoIP. Could implement browser to server
  • Instant messaging
  • User to user(s), friend to friend(s), instant messaging like skype, messenger, fb chat, jabber. Guest restricted to admin users
  • Online and offline messaging between users or groups of users chat room style
  • Support jabber servers like fb chat. Can chat externally via XMPP
  • Messages can be either private or public. New participants cannot be added to private messages
  • Message history preserved like emails
  • Used in conjunction with webRTC in modern contact pages to replace contact form
  • Messages can be titled and associated with a product or event. HW use to ask questions about pieces of art
  • Messages can be multiuser chat rooms with a theme
  • Private social media
  • Facebook style user interactions around a status, event or product context
  • Private friends feed or public newsfeed views
  • Comments, likes, dislikes, post as you or post anonymous
  • Private image uploads for users
  • Guests can like, comment and interact but can see only public postings with restricted commenting
  • Statuses can be given a product or event context and auto created. Winners tiles and art piece interactions
  • POSSE. Post once then syndicate around public social media. Auto create social media posts from events to twitter/facebook
  • Live connections and monitoring
  • Monitor connections, users, ip addresses live. Entry, exit, how many visits, stay length
  • Historical connection monitoring. Daily, weekly, monthly, user to guest ratio - updated live
  • Maximum connection restriction. Twitter is over capacity style...
  • Who is doing what on the site. Our CashClamber demo shows live user activity
  • Live user website language translations
  • All our static webpages utilise token substitution dynamically on webserver HTTP GET
  • Even the default 'English' language is token substituted on each HTTP request
  • Website can be only available in 'English' or any language that it has been translated into
  • Users or special translator users can offer translations live on the website. Admin user authorises a translation
  • Authorised translations immediately appear live on the website
  • Translations are managed in the framework database. However fast token maps are written and used
  • Website translations are handled by the website live out of the box
  • Great for multi-cultural societies or worldwide access
  • Laptop, tablet and mobile versions
  • Utilise CSS media queries to style the web page for best use of display size and orientation. Responsive design
  • One source code base. No need for mobile apps or tablet apps. Webapp just scales to best fit
  • It's possible to create a browser style app to add to the iTunes or play stores for download. For user search visibility
  • Javascript obfuscation
  • We obfuscate all our client side JavaScript source code. Initially with google closure. We also have our own advanced process
  • We code our JS to a strict declarative commenting standard. We identify functions, variables...
  • Our obfuscator can then confidently shorten JS programming element names, functions, variables reducing size/name obfuscation
  • We combine our multi-module JS files into a single shortened download file. Can get 50% more saving over closure alone
  • Our client side JS is unreadable, unsearchable, completely tokenised and unable to run in the browser debugger
  • Since we use our own obfuscation technology it's not in the public domain so reverse engineering is tough
  • Paths to server side scripting files are hidden from searches. Any PHP helpers in particular
  • We generate index mapping file to help us find runtime JS issues
  • Security
  • Tradition security associated with any web server
  • Javascript obfuscation described above helps hide the structure of the server directories
  • How to test security. Simply put up a bitcoin website like CashClamber in the public domain. Sit back and wait
  • Websockets have their own perculiar set of security issues. It's an open socket listening which receives data
  • Websockets out of the box are cross domain. A script hosted at another website can connect to our websocket and interact
  • Our websocket server is NOT cross domain. We allow only connections from our domain or allowed domains
  • We have an additional soft handshake during connection which ensures the only valid connections are our browser connections
  • Our soft handshake includes contains a realtime generated key. Different each connect. Stops wireshark traffic monitoring
  • We run our own websocket server implementation. Node.js has security vunerabilities and exploits. Our security is unknown
  • Currently we send unsecure binary plain text messages. If ws client implementation is text plain text messages
  • All message originate from our allowed domains so messages can't be spoofed from another domain
  • Although we haven't yet implemented there's wss secure connection just like https. Or encrypt and send over http
  • 99% of our server code is native. Much harder to read than regular scripting files
  • If we were hackable we would have been hacked. We've been constantly and are being constantly attacked
  • Error handling and support
  • In our server framework each thread is exception safe. Any errors do not crash the application just terminate the thread
  • Our runtime server code contains a code debugger. Each thread exception creates a stack trace down to the source file and line
  • Any SQL errors handled the same way as thread exceptions. Erroneous SQL statement is created
  • User generated Javascript browser errors are centrally reported and logged
  • Exceptions and bad SQL are handled as admin alerts. Each admin is messaged and send the error report
  • We can find difficult errors really quickly. Ever tried a coredump and a WER restart. That's why the above exists we have
  • ALL errors are centrally reported to our support dashboard in realtime
  • What's CashClamber?
  • A demonstration of all our html5 live framework features. Everything we do and add is in here first
  • A bit of fun with a serious side. Testing. It's how we effectively test our latest live features
  • Unit tests, smoke tests, all extreme programming paradigm is good. But human stress testing is hard
  • We are a small team without a QA department and testers. This helps us a great deal. Free, or low cost, diverse human testing
  • We employ 1000's of testers worldwide for bitcoin. Each gamer is playing for bitcoin but indirectly testing our latest features
  • Our error handling alerts feed back any exceptions or bad SQL down to the line of code
  • Visit http://cashclamber.com
  • Framework enhancements improve existing applications
  • Our applications are not fixed at the point of production, both client and server side code inherit from the framework
  • New framework features, tested and proved in CashClamber, are automatically included in successive builds of application types
  • New application features for specific project, are generalised if useful, and feedback into the framework and other applications
  • All framework applications continue to grow with improvements in our framework
  • Reusable common forms and controls
  • A library of common UI/UX elements that can be reused on different application types and projects
  • Our UI controls are implemented both in the client JS and Server application core using messaging to interact
  • UI controls are object orientated both in JS and server code for further derivation without code duplication
  • All UI elements are skinned using CSS so the same control can appear cosmetically different per application type
  • Special live and shareable controls are available. Handle UI concurrency out of the box. Share across multiple dialogs
  • One more small step to move our controls shadow DOM, imports and templates. Stock controls as web resources. Self containment
  • Scalability
    Application Instance Manager
  • Allow more than one application instance to run on the same server. All with unique friends, newsfeed, data...
  • Each application instance has the same webserver html but different wording and skinning. Look different
  • Example. Host multiple shops for the same company with the organisation. Host multiple tailored clients on the same hosting
  • Allows a client to reuse the same VPS for multiple clients. Sell the application as a server on a subscription model
  • Allow an superuser administrator control over the running instances. Start, stop, create easily
  • Allow disparate types of application instances. Host multiple applications types and instances on the same server
  • Redis
  • Websocket connections are sticky. Web server connections are not sticky (to a point). Web farm is easier for web server
  • Multiple web servers behind load balancing switch(s) must route messages to the same intial connection server. Configuration
  • A server farm must distribute a routing directory. A map of connections to servers and re-route messages
  • Shared single redis server in redis layer (DMZ2) which routes messages via a directory of WS layer (DMZ1) connections
  • Multiple load balance redis servers in redis layer with shared memory or DB directory lookup
  • Clustered LB redis servers in redis layer. Each has connection directory copy mirror
  • Support dashboard
  • Project Peach live website. Our management software running on top of all our live realtime sites
  • Dashboard monitoring of all our servers, AIM, Admin panels. Access to all AIMs and admin panels
  • Using allowable cross domain websocket messaging live monitoring of every webapp we create in realtime
  • Ultimate destination for all admin alerts. Exceptions and errors are sent here. Fix client problems before bug report
  • Central point of communication between Project Peach and our client admins. VoIP, IM, remote desktop in webapp
  • Heartbeat and system health constantly monitored. Support can support effectively
  • And in 2017?
    We've not been idle since 2014. We've been constantly building both technical architecture, infrastructure and both writing / refactoring our code base constantly. We've fine tuned and created vertical packages like our live eCommerce solution. There's just too much to detail here. If you're as forward thinking as the Met Office were back in 2014, we'll blow you away with a demo of our 2017 live webapp framework. Want to help us grow? We're all ears!
    ##BLOGWRITTENBY##
    John Ince
    Comments
    ##GUEST##
    01 Jun 2017 15:33:22UTC
    how many hacks have you seen off through the bitcoin site?