Reblogged from Four Nines:

Web sites and web applications are increasingly using secure connections (HTTPS) for all traffic not just obviously sensitive data, as a way to guard against security threats. However, HTTPS requires encryption/decryption of data, which is computationally intensive. Web applications can therefore benefit from “offloading” the encryption/decryption processing required for HTTPS to specialised hardware devices. …

Originally posted at Four Nines (http://fournines.wordpress.com/2011/12/08/speed-up-web-applications-with-ssl-offloading/)

Is extreme high availability a bad thing?

• “…having a higher degree of PER ELEMENT failure, while allowing an architecture to get you to extremely high overall reliability, is one of the more transformative features of many cloud options…“
• Several comments about how reliability is less immediately appealing to businesses  than new features, but of course reliability is the slow-burning coal, whereas new features are often just so much kindling (and therefore “burn out” very quickly).

What are you using for server and network monitoring?

• Zabbix and Nagios are the usual suspects, with Splunk in the mix too (obviously not a direct comparison), with Zenoss making waves. Little input from anyone running SCOM for Windows; presumably these folks are using Zabbix or Nagios(?!).

Literature on website scalability

Some useful print and online resources for building reliable websites, including:

• Seven Databases in Seven Weeks
  • http://pragprog.com/book/rwdata/seven-databases-in-seven-weeks
• High Performance Web Sites[although this probably needs updating now to include WebSockets, Node.js, etc.]
  • http://shop.oreilly.com/product/9780596529307.do
• Gigaspaces XAP architecture overview
  • http://www.gigaspaces.com/wiki/display/XAP8/Product+Architecture
• Experience with some Principles for Building an Internet-Scale Reliable System(Akamai – PDF, 160 Kb)
  • http://www.akamai.com/dl/technical_publications/ ExperiencewithsomePrinciplesforBuildinganInternetScaleReliableSystem.pdf

Details

CLR-to-Native Win32

The CLR subsystem responsible for managing access to the native platform is known as P/Invoke. These services are included in an application by use of the namespace System.Runtime.InteropServices, and the [DllImport(“<DLL_NAME>”)] attribute prepended to the function prototype.

When a call goes out to a piece of Unmanaged code from Managed code (CLR), a flag is set on the application’s CLR (pseudo-) stack. This causes the GC not to collect during the duration of the Unmanaged call. The Security Subsystem responds to the flag by searching up the stack for permissions to enter Unmanaged code. The security check can be suppressed by a call to System.Security.SuppressUnmanagedCodeSecurity, which prevents stack crawling by the Security Subsystem beyond the point at which the call was made.

P/Invoke builds a second stackframe to use when calling the Unmanaged code. Some types can be blitted from the CLR stackframe to the Unmanaged frame, e.g. integers. Other types such as System.String are less easy to transfer to the new frame; instead of a reference to the string memory being passed, a char*-compatible copy is made, meaning that the intended manipulation of the character string inside the Unmanaged code does not get transferred back to the CLR string.[1]

P/Invoke performs type conversion/coercion automatically on-the-fly. The precise native (Win32) type of converted CLR types can by controlled using the MarshallAs directive.

Note that a call through P/Invoke is expensive: a minimum of 32 machine instructions is required for the Managed-to-Unmanaged transition. In practice this figure will be much larger.

CLR-to-COM

The CLR can interop with COM. CLR types appear as COM objects to COM via CCWs, and COM objects appear as CLR types to the CLR via RCWs. The tools TLBIMP.exe and TLBEXP.exe can be used to generate CCWs and RCWs.

ComImport is a compiler directive (== metadata property) that directs the CLR to create the object to which it refers using the classic COM CoCreateInstance rather than from its own GC-managed heap.

COM-to-CLR

The tool RegAsm.exe registers types found in a .NET assembly as COM objects. These COM objects are hosted in mscoree.exe as InProc servers.[2] mscoree.dll and mscorlib.dll are the CLR COM Type Libraries.

Comments

This document is based on a lecture given at DevWeek 2002.

Glossary

• CCW……………………………………….. COM-Callable Wrapper
• CLR………………………………………… Common Language Runtime
• GC………………………………………….. Garbage Collector
• P/Invoke………………………………… Platform Invoke
• RCW……………………………………….. Runtime-Callable Wrapper

Details

Method Calls

Every object in the CLR has type information associated with it, in the form of a 36-byte header:

Every method on the object has an entry in the method table (which acts in a similar way to a vtable in C++). Each entry is in effect a function pointer, and all method calls on the object come via the method table.

Immediately following object construction (using the .ctor or .cctor method) the objects method table appears thus:

The entry in the object’s method table points to the first element of the Pre-stub worker (the Instruction Pointer). This in turn has been initialised to point to an execution block in the JIT compiler. The JIT compiler then dereferences the second element (the IL Pointer) of the Pre-stub worker to read the IL code for the requested method from the Assembly where the method resides.

The JIT compiler then JITs the IL code to produce native machine code (e.g. x86 instructions), and then redirects the first member of the Pre-worker stub to point to that native code instead:

Following JITting, the method pointer has only a single indirection in the stub before reaching native code. This is actually better than native Win32 DLLs, where, due to base address relocation, there is usually a second indirection.

Interface Method Calls

There is no obvious order for interface method implementations in the method table; unlike the situation for classes, a derived object can implement the methods of a ‘higher’ interface than its ancestor. Therefore, an extra level of indirection is needed to map the method table entries of interface methods to their implementations:

Dynamic Code Execution

How to execute code based on user input, e.g. “Execute the method called ‘FooBar’” is solved in a much more efficient way than under COM, where the IDispatch interface with Invoke was notoriously slow.

Under .NET there are three basic options:

1. Interface and case statement

Devise an interface (e.g. IDynamicExecute) and have the implementing class instance check in a case statement the requested method name against the list of methods it implements.

Disadvantage: no compile-time type checking.

2. GetMethod, Invoke

Use GetType.GetMethod(<methodname>) and <method>.Invoke. Use the MethodInfo type to return a list of methods by signature (but not return value).

Disadvantage: about 1000× slower than standard method calls, due to run-time type-checking.

3. Delegates

To grab a function pointer, Method.MethodHandle.GetFunctionPointer can be used, but IL assembly code is required to make use of this.

Use Delegate.CreateDelegate to create a method variable dynamically bound to a specific object instance. Delegate methods are marked with the runtime modifier to show that their implementations will be created on-the-fly by the CLR when the delegate object is created. The code they execute is specified at runtime.

Execution overhead is small, with only eight extra machine instructions; the ecx register (this pointer) is flipped by the CLR to point to the code to execute.

Asynchronous Methods

Asynchronous method calls use Delegates. Events allow [un]registration for specified conditions, and are also implemented using Delegates. Threads from the CLR thread pool are used to fulfil the async request[1].

Rules developed for async method calls in COM+ and MTS apply here.

Comments

This document is based on a lecture given at DevWeek 2002.

Glossary

• vtable: virtual method table
• .ctor: constructor method
• .cctor: class (static) constructor method

Details

Definitions

A Context is a way to group together CLR objects having similar runtime (execution) requirements. Contexts are created as needed by the CLR. Cross-context calls require proxies, in a similar manner to cross-Apartment calls in COM.

Execution requirements could include:

• Synchronisation
• Transactional support
• Database updates

Contexts also allow arbitrary message processing to be ‘plugged-in’ to the method-processing architecture by the use of Proxies.

Example

CLR Process 1 contains a single AppDomain, which in turn contains two Contexts. Object C must be marked with special processing requirements (say, Interlocked access), and so is placed in a Context which supports this feature. The default (first) Context has no special processing features.

Objects A & B were marked with no special requirements, and so were placed in the Default Context. In order to access Object C, Object B must call through the Proxy. To B, the Proxy appears just like C; in fact B does not know that a proxy is being used. Likewise, if C needs to call A, it uses a Proxy which resembles A in terms of method signatures etc.

Context Creation

The CLR creates Contexts automatically as required: if the Context attributes of the current context do not fit the attributes of the object about to be created, then a new Context is created, and the object placed in there.

Sinks & Properties

Sinks and Properties are entities within Context Proxies which allow interception of cross-Context Object access, and the ‘plugging-in’ of arbitrary additional or replacement processing of method calls. For example, a Sink may decide, based on some security settings, to refuse to pass on a method call to the real object and simply return False:

• Sink  - read-write access to cross-Context messages
• Property – read-only access to messages

A typical use of a Sink would be to acquire and release a Synchronisation lock around a method call:

This means that the client object () does not need to know anything about synchronisation locks, and can be re-used on a scenario without them, or indeed with some other type of processing around the method call.

Comments

This document is based on a lecture given at DevWeek 2002.

Glossary

Context-Agile Object ………………… object refs can be passed between Contexts

Context-Bound Object ………………. object access between Contexts is via a proxy

AppDomain ……………………………… logical process space within the CLR

Summary

This note describes a simple but flexible error-reporting and tracing framework for Delphi 4 and above based on Assert, which provides the unit name and line number at which errors were trapped and traces made.

Details

Background

Under the Delphi Language there is no simple way of replicating the C/C’++ macros _—FILE_— and _LINE_ to obtain the unit name and line number of memory address at runtime. However, in his paper “Non-Obvious Debugging Techniques” Brian Long points out that the Delphi compiler provides both unit name and line number during a call to Assert, and describes how assertions can be exploited to provide detailed execution tracing.

The framework described here extends this idea to allow flexibility in the processing of assertions. Assertion processing can be switched on and off at runtime; arbitrary filtering can be applied to any assertion; and both execution tracing and ‘standard’ assertion behaviour (i.e. raising an exception) can be effected. Assertions can therefore be left enabled in production code, at the expense of a slightly larger binary executable.

Design

If SysUtils is included in the uses clause of a unit within a project, the default Pascal-style Run-Time Error (RTE) error handling of the System unit is replaced with one based on exceptions. By default a failed assertion causes RTE 227, but with SysUtils an EAssertionFailed exception is raised instead, which normally just displays a message box:

procedure TForm1.Button1C1ick(Sender: TObject);
 begin
Assert(False, ‘Inside Button1Click’);
end;

The default behaviour is changed by reassigning the AssertErrorProc variable to a custom routine. The failed assertion, along with all the unit and line number information, can then be extracted and passed to a logging or error recovery mechanism.

The library unit AssertLib.pas redirects assertion handling to its own custom routine; when an assertion fails, the message string is checked for special substrings, and the assertion logged or an exception thrown as appropriate. Assuming a hypotheical Assertion object, the general algorithm is:

// assertion handler routine
 it IsDebugString(Assertion.Messaqe) then
  LogDebugString
else if IsSevereError(Assertion.Message) then
   raise ESevereError...
 else
   raise ENormalError...;

The exception handling of the application or DLL is relied upon to deal with the exceptions raised. Crucial here is that the unit name and line number are extracted before the exceptions are raised, and stored in custom fields of the exception objects:

type EAssertDetails = class(Exception)
public
constructor CreatedDetailed( const Msg, AUnitName : string;
AlineNumber, AAddress : Integer);
   property UnitName : string read FUnitName write SetUnitName;
   property LineNumber : Integer read FLineNumber write SetLineNumber;
   property Address    Integer read FAddress write SetAddress; end;
EAssertDetailsFatal   = class(EAssertDetails);
 .
 .

 // inside the assertion handler
 else if IsSevereError(Assertion.Message) then
raise EAssertDetailsFatal.CreateDetailed( Assertion.Message, UnitName, LineNumber, ErrorAddress);

If the logging/exception framework is aware of these special fields, it can extract the information easily, and tailor its behaviour to the way in which Assert is being used:

procedure TForml.HandleException(Sender: TObject; E:Exception);
 var temp : string;
begin
if E is EAssertDetails then
begin
  // Extract unit name and line number from special exception object

 temp := Format (Sender %s reports exception: %s. + Unit: %s, Line: %d, Address: %d.’, [Sender.ClassName, E.Message, E.UnitName, E.LineNumber, E.Address]);
// Notify the user with a message dialog if required
   if E is EAssertDetailsNotify then ShowMessage(temp);
// The assertion was deemed fatal, so terminate the app
 if E is EAssertDetailsFatal then
ShowMessage(temp);
Application.Terminate;
end;

 // Add other specialized descendents of EAssertDetails as required

 end
else
temp := E.Message;

 // Log all errors
 Memol.Lines.Add(temp);
 end;

Usage

Assert can be used to pinpoint error locations whenever it can be assumed that assertions are turned on ({C+}). For example, low-level code and support classes should not use assertions for error reporting, because they cannot rely on assertions being enabled. In general, if a section of code expects flow to be broken in an error condition, then exceptions must be used as normal.

User interface code, on the other hand, can usually use Assert in safety, because the exceptions raised in the custom assertion handler will be handled by code at the same level (probably with Application.OnException). Also, User Interface code can trap exceptions raised at a lower level, and then use Assert combined with Delphi’s Run-Time Type Information to report the exception:

procedure TrapException;
var i : Integer;
begin try
i := 10 div 0;
except
on E: Exception do
Assert(False, E.ClassName + ' ' + E.Message);
end;
end;

In this way, most of the information provided by the exception is retained (e.g. the class reference is lost), but we gain information about which section of code was executing when the exception was trapped.

Advanced Features

Because failed assertions are routed through a custom routine, their behaviour can be controlled at runtime. The GAssertEnabled flag in AssertLib determines whether a failed assertion should be acted upon or ignored. This behaviour is dependent  upon the instantaneous value of GAssertEnabled, so can be switched on and off for different sections of code.

As alluded to above, Assert can be used for debugging traces. By prepending the constant S_—DEBUG_—TRACE to the assertion error message, and forcing an assertion failure, line-by-line traces of a code path can be logged.

The custom AssertErrorProc checks for S_DEBUG_TRACE and passes the assertion information (including line number etc.) to a special debug trace procedure, set using SetDebugLogProcedure. In this case, an exception is not raised, so that the logic flow is not broken in the traced routine. Acknowledgement of these trace assertions can be controlled independently of other assertions, because it is assumed that trace assertions may be much more common than ‘error’ assertions. The flag GRecogniseDebugAssertions controls this behaviour:

procedure TraceMeBaby;
begin
  Assert(False, S DEBUG TRACE + ‘entering TraceMeBaby...’);
   // do some work
   // ...
   Assert(False, S_DEBUG_TRACE + ‘...leaving TraceMeBaby’);
 end;

By defining other special prefixes and prepending them to assertion messages, the custom AssertErrorProc can distinguish further uses of Assert.

S_—SEVERE_—ERROR is one such indicator; it is used to report severe but foreseen errors in program operation.

Limitations and Improvements

Because Assert is the only easy way to access unit name and line number information in Delphi, such information is only valid for the place where Assert was called (contrast this with the C/C++ macros __FILE__ and __LINE__ which can be used anywhere). The location of the Assert call may not correspond to the true location of the error, but rather to the point at which the error was detected. Until Delphi acquires native functions equivalent to the C macros, this limitation (and the need for this Assert-based framework) will remain.

Control of assertions could be made more extensible by being given a class wrapper, with the concept of registering different assertion conditions:

// Assertion class to call MessageBeep() on assertion failure
 EAssertDetailsBeep = class(EAssertDetails);
// Register new assertion class, using prefix identifier string and class reference
 AssertionManager.RegisterAssertionClass(‘[B]’, EAssertDetailsBeep);

‘This would allow arbitrary classes deriving from EAssertDetails to be defined outside AssertLib.pas; the AssertionManager would search a list of identifier prefixes for all registered assertion classes and raise the appropriate exception.

Another improvement would be to allow per-thread control of assertion behaviour, by marking switches such as GAssertEnabled with the threadvar keyword. This would solve some concurrency issues that would otherwise occur if the framework were used in a multithreaded application.

Conclusion

The ability to trace detection of an error to a specific line in the source code is very useful, but fully-featured frameworks to achieve this are by necessity complex. The framework described here requires no changes to existing code, other than AssertLib to be initialized after SysUtils, a small number of variables to be set, and assertions enabled.

Drop Test by Christoph Bauer - http://www.flickr.com/photos/h34dy/

1. White (built atop the Microsoft UIAutomation library - free ($0): http://white.codeplex.com/ but roll your own test harness. This is a ThoughtWorks project, so pretty high quality. Some usage examples here: http://blog.benhall.me.uk/2008/02/project-white-automated-ui-testing.html
2. Test Automation FX: http://www.testautomationfx.com/ - free ($0) and commercial versions (~ $300)
3. TestComplete: http://smartbear.com/products/qa-tools/automated-testing/ - between $1000 and $4500 per license
4. HP Quick Test Pro (QTP): http://www8.hp.com/uk/en/software/software-product.html?compURI=tcm:183-936981 - $$$ but does web testing also.

Since it’s been several years since I built any serious systems using WinForms or VC++/MFC, I cannot really comment on the effectiveness of any of these, but  an ex-colleague of mine tells me that although TestComplete is okay (and good value for money), it has its problems: poor product support, and apparently it intermittently does “weird things” like “stuff just randomly stops working” (never a good sign).

It seems like using the Microsoft UIAutomation library (as White does) is the best bet at the moment; it’ll be interesting to see what comes out as WinRT becomes more adopted, as WinRT avoid many of the problems of the existing Win32 API, and therefore automating desktop applications should (I hope) become simpler and less error-prone (the current Window message-queue implementation is pretty long in the tooth).

My prediction is that by 2015, it will be bad practice to access web sites via HTTP, and users will increasingly demand HTTPS. This has interesting implications for hardware manufacturers and software development teams alike.

Introduction

Public WiFi connections – such as hotspots in coffee shops, trains, airports and libraries – are often unsecured out of necessity, leaving their connections open to being hijacked or “sniffed” by malicious people using freely-available tools such as SniffWifi, Kismet and NetStumbler.

To protect users from data and privacy loss due to insecure and untrusted connections, web sites and applications are increasingly being run entirely over secure connections (HTTPS – the so-called “green address bar”). Google, Facebook, Twitter and Github are examples of popular websites offering (or forcing) the use of HTTPS to users on the site.

The popular source code collaboration site Github serves all content via secure HTTPS

My prediction is that by 2015, internet users will expect a secure connection when accessing every website and by extension every type of content. Using HTTP is analogous to writing postcards: everyone can see the message as it’s sent across the communication network. Only the most trivial of messages are written on postcards; for all other correspondence, sealed envelopes are used for privacy.

Implications of Ubiquitous Secure Connections

The implications of “HTTPS Everywhere” seem to be:

1. Hardware acceleration of HTTPS encryption/decryption will provide a speed advantage
2. Use of response headers and other settings will need to be improved
3. The deployment and revocation of SSL/TLS certificates will need to be increasingly automated
4. Wildcard certificates and turn-key single sign-on systems will become more popular

Dedicated Encryption Hardware and SSL Offload

Dedicated hardware is generally faster than an equivalent software implementation (for any algorithm). Innovations such as Intel’s AES-NI (Advanced Encryption Standard New Instructions) can provide a speed advantage for client computers because the encryption/decryption needed when communicating over HTTPS can be greatly accelerated.

Likewise, at the server side, hardware with dedicated SSL/TLS-handling hardware will outperform software-based or virtualised SSL/TLS routines (such as when IIS or Apache handles HTTPS termination, or when a virtual appliance is used as a content switch).

.

Improving Use of Response Headers and Other Settings

Given that, in general, HTTPS connections are slower than those over HTTP, due to the extra time needed to encrypt/decrypt the data, any web sites which take advantage of HTTP Cache Control Headers and other HTTP response headers in an intelligent and relevant way will improve the experience for the user (although there can be some security implications for some older browsers, which can cache HTTPS content to disk (!)). In particular, text content which has Gzip compression applied on the server before being encrypted for HTTPS will often arrive at the browser more quickly than if it were not compressed.

Automated Certificate Deployment and Revocation

It will become increasing important to be able to both deploy certificates and revoke certificates using automation. Automatic provisioning of new certificates for HTTPS is relatively common already, but the importance of timely certificate revocation in the event of a break in the chain of trust will be vital for a company’s reputation (such as when a root CA has been compromised).

Wildcard Certificates

So-called “wildcard” certificates are digital certificates which can apply to more than one server address, so instead of a certificate being valid for (say) secure.domain.com, it is issued for *.domain.com (hence the name “wildcard”).

As more and more users expect all communication to be secure, wildcard certificates will be increasingly useful to website owners, because a wildcard certificate will cover all parts of the website, such as images.domain.com, login.domain.com and www.domain.com, all of which will need to be delivered by HTTPS to avoid “mixed content vulnerabilities“.

Even Google doesn’t always avoid “mixed content”

The feared (and confusing) “mixed content” warning in IE8

Without a wildcard certificate, a website owner would typically need to buy individual certificates for each subdomain separately, a significant cost.

Summary

I cannot see how the spread of secure (HTTPS) connections will do anything other than increase in the next few years. This has some interesting implications for those owning, building and hosting websites, as they will need to raise their game in terms of encryption/decryption speed, content caching and certificate management.

[Posting this here, as the only comments which seems to have been approved on the OP itself are those talking abut how "cute" the interviewer is - epic fail]

Paul’s points in the interview about making use of I/O completion ports on Windows highlights a key issue in cross-platform software development: Windows has some incredibly powerful and advanced APIs, which – if used directly – can provide huge performance benefits for software that uses them.

The common model of “develop for Linux first, then port to Windows” can mean that the ported software performs poorly on Windows installations, at least partly because you’re imposing a Linux-y execution model on a Windows system. You only need to look at the number of application handles opened by 32-bit MySQL running on Windows (at least 32k handles at startup – for *what* reason, exactly?) to see an example of a such a non-idiomatic Linux-to-Windows port.

The spoken language analogy would be using Google Translate for all your i18n needs: a big fail. Paul also points out that there is a large installed base of Windows servers out there which are not going away any time soon, so ignoring Windows when developing software is somewhat negligent.

In fact, Node in effect re-implemented I/O completion ports (http://msdn.microsoft.com/en-us/library/windows/desktop/aa365198(v=vs.85).aspx, “an efficient threading model for processing multiple asynchronous I/O requests on a multiprocessor system“) inside the Node code, effectively making them available on Linux too.

Under the cover of all the Windows marketing/sales hype, there is a [largely-] POSIX-compliant, well-architected OS with some features (such as I/O completion ports) which Linux lacks. Linux obviously has features (User-mode Linux, for instance) which Windows could really use; my point here is to highlight that the “Linux first, then transliterate for Windows” is missing a trick or two.

The take-home points were emphasised in the last 15 mins when James Lewis ran through details of the AWS outage in April this year which brought down video streaming service Netflix:

• The cloud can be incredibly flexible and rapid for provisioning
• The cloud hides much of the complexity of traditional data centre configuration
• However, we still need to design for failure (as we always have)

The root-cause analysis by Amazon makes for interesting reading (as it goes into some detail about how their cloud services work). Ultimately, though, they make the point that:

if your systems failed in the Amazon cloud this week, it wasn’t Amazon’s fault. You either deemed an outage of this nature an acceptable risk or you failed to design for Amazon’s cloud computing model.

This is why software developers and architects still need to understand failure modes, resiliency, redundancy and so on, and design their software to expect failure and deal with it.

“The Cloud” does not absolve software developers from planning for failures, and four nines (99.99%) uptime is still hard to achieve.