How To Square A Machine Tool

Squaring your machine tool

What Do You Mean “Square”?

This month we will discuss squaring a machine tool. There are many trains of thought in the machine metrology world on the best way to measure and square a machine. By squaring a machine we mean to verify the X, Y, and Z axis are traveling at 90 degrees to each other. This way, if we cut a square, we will get a square that is square and not a trapezoid or polygon. Also, strangely enough, if we cut a circle and we are not square, we will not get a circle, but an ellipse. This being said, many metrologists and technicians prefer to program a circle in order to measure and square a machine. I beg to differ.

How Do I Square Thee? Let Me Count the Ways …

Squaring a machine tool may sound as simple and basic as putting a nut on a bolt, but there are many factors and misconceptions surrounding this essential event. The X, Y, Z perpendicularity of a machine is the foundation upon which all other calibrations are based. There are many methods to measure the squareness of a machine. Here are many of them:

  • Tracker laser
  • Tracer laser
  • Laser interferometer
  • Laser and pentaprism
  • Ball bar
  • Tetrahedron and probe
  • Probe and artifact
  • Traceable artifact and indicator, and
  • Machined parts measurement

Let’s break down each of these methods and discuss their pros and cons.

Tape Measure for Tooling

Method: Machined Parts Measurement

Starting with the last option listed above, we may machine parts and then measure them.  I like to start with this choice because this is the final word on a machine’s capabilities, the ability to make accurate parts. The beauty of this method is everyone with a machine tool has the ability to perform this test. The drawback is the level of measurement precision. For a cabinet shop, sawing or machining a square and measuring it with a tape measure may be all that is needed to verify machinery and make good parts. For a manufacturer of fighter jets or missiles, whose parts may be many meters in length and width, and need to fit within a tenth of a millimeter or a few thousandths of an inch, a tape measure just won’t do. We won’t get into this today but it is these industries that are the driving force behind the developments in large scale metrology, which are taking place. Big parts, thin flexible parts, and these coupled with the need for high accuracy, are tough critters to catch hold of in the measurement world.

Circle/Diamond/Square – CDS

But let’s get back to the machining and measurement parts. One of the common test parts cut is the circle – diamond – square.  In this test a suitable, stable, material is chosen and a square is machined as a base, then a circle is machined on top of it, and finally a diamond is machined on top of the circle.  There are many variations for this method, and many include drilled and milled holes on the top and/or sides of the test part along with angled milling.  The finished part is then CMM’d, or measured somehow, and the discussion begins. This is a very good test, but with some drawbacks. For the technician the difficulty is in how to determine which element involved is the troublemaker, if the test shows something is out of tolerance. For the manufacturer who builds large parts a 12” CDS (circle-diamond-square) doesn’t really prove much.

You Still Have to Come Back to the Basics

Since CMS’s main interest is in large envelope machines, let’s move on to other methods of proving out a machine’s geometry. Let’s say a CDS was cut and failed some portion of the test. The technician now has to find out why.  It’s always in the best interest of the technician to lift and separate, or is it divide and conquer, the problem. Either way we must isolate all the individual factors that could be introducing the inaccuracies.  So we’re back to the examining the basic foundations of the machine: the X, Y and Z squareness relationships.

Method: A Group with Similar Pros & Cons

Next, let’s lump the following methods together; because these methods are basically the same with regard to their pros and cons:  Tetrahedron and probe, probe and artifact, traceable artifact and indicator. All three methods can use a tactical probe or a dial indicator. The probe allows for automation and speed but requires programming; the indicator allows for an operator or mechanical technician to perform the tests but requires operator level skills for good results.


The tetrahedron is basically a pyramid with tooling balls at the 5 points of the tetrahedron: four at the base and one at the top center connected with thermally stable, usually carbon, rods at a certified distance.  These balls are probed or measured, and then either manually compared to the known dimensions or run through some software to determine the condition of the machine.  The balls are available in sizes of up to a meter and are a very good method of checking a machine, especially if your parts are near the size of the tetrahedron.

Probe & Artifact

The probe and artifact are very similar to the tetrahedron, but the artifacts are usually attached to the machine, whereas the tetrahedron is portable.  The artifacts can be either bushings or balls, but the certainty of their location must be absolute. A minimum of 3 points are needed and the further away from each other they are the better.  I generally prefer 4 points: one at each extreme of the X and Y travel for the first 3 points then the 4th at the extreme point of the Z travel.  So these points would be located at the front top left, front top right, rear top right and front right bottom. These measured points can be processed through software and defined as 3 planes and their relationship compared to each other, or just checked against a programmed tolerance window, or manually examined.

Traceable Artifact

The last method is the traceable artifact, such as a granite square or ceramic square, and a probe or indicator. This is also a viable method but is usually restricted in size. How can you square a 7500 mm x 10000 mm machine with a 200mm square? The biggest pros for this method are cost, speed and skill level needed.  Every shop should have these devices at a bare minimum.

Method: Laser Tracer

It’s now onto the fancier electronic and laser options of measurement. The current best method of squareness measurement is through the laser tracer. There are two companies worldwide, I know of, that offer this solution. The principal, which supports this method, is using a laser interferometer which follows a special mirrored reflector mounted to the spindle and measures the distances to a grid work of points. The tracer is then moved to a different location and the points remeasured. This is repeated a minimum of 3 times, but the more the merrier.  The data is then introduced into software which will triangulate all the points and give you a map of the machine.  The accuracy of this method provides the repeatability of the machine; the measured points are accurate to a nanometer. The equipment to perform this test costs about $200,000 (US) and takes about a day to gather the data depending on the machine size.  Small machines may only take an hour.  Through other software a compensation map can be produced and downloaded into some controllers and a full 3 axis volumetric compensation will be achieved.  Currently to hire this done costs around $45,000.

Method: Laser Tracker

Next on the list is the laser tracker.  The tracker, like the tracer, will follow a reflector attached to the spindle but the tracker produces a 3 dimensional measurement at each point.  It uses 2 encoders for azimuth and declination and an interferometer for distance. The accuracy of these devices is about 0.001” in 20 feet. With the use of artifacts, or control points, the tracker can be moved many times without the loss of accuracy.  With the proper use of this equipment the amount of data, the precision, and the speed at which it can be collected is incredible.  The cost of this equipment can be as low as $80,000 and as high as $300,000.  There are 3 companies worldwide that make these devices and all have two models. There are other methods using multiple trackers and using triangulation and/or trilateration but the cost continues to climb.

Method: Laser and Pentaprism

Next there’s the Laser and pentaprism. The particular type of laser I’m referring to is the steady beam or rotating beam laser and a CCD or light sensing diode. These devices are good for 3 things: straightness, flatness and squareness.  CMS owns around 10 of these. These devices measure geometry and geometry only, and measure it well. The cost drops to $20,000, on the low end, to $60,000, on the top. The operating principal of these is a beam of light is produced and sent flying down the machine. On the other end is a target or receiver with a very sensitive light sensor in it. The target is attached to the machine and moved along the axis. As the target moves up and down and/or side to side the sensor reads where the beam is hitting it and correlates it into a distance measurement.  For squareness the beam is aligned with one of the axes and then turned 90 degrees through a pentaprism and the second axis is then measured. A pentaprism is a fancy mirror that turns the beam 90 degrees regardless of the angle the light enters the device.  They are usually ground to a precision of within ½ arc second. That will give us about 2 microns per meter or 0.001” in 30 feet.  The drawback of this equipment is the effect the environment has on the beam. Beyond 20 feet the measurement gets more and more unstable and moving the laser is not an easy task. As a final note on the pentaprism. They are beautiful little pieces of shiny glass and like the diamond they resemble they are not cheap.  The pentaprisms CMS uses cost $8500 each.

Method: Interferometers

Interferometers are next on our list of geometrical toys. As we all know, interferometers are really fancy tape measures. They have the ability to resolve down to 1 nanometer: one millionth of a millimeter.  As mentioned earlier, the cabinet shop may cut a square and measure it with a tape measure and say, “Hey, that drawer fits fine!”  Now we have the ability to measure the diagonals of a square to a nanometer. With that ability I’d say, “Hey, that drawer fits PERFECT!”  This is currently my favorite and only way I will square a machine.  An important note here though, the machine must be straight. This method only measures the total diagonal length it does not see any dips, wobbles, or humps in between the end points. Also the machine must be distance calibrated. For example, if you have a master-slave axis, as in a gantry bridge machine, the master and slave must move exactly the same distance. We can have a trapezoid but not a polygon.  Both opposing sides must be the same length. Lastly, if the machine is very “over-square,” it’s not good practice to measure the extreme corners.  By over-square I mean the Y axis may be 1 meter and the X axis 20 meters. In such a situation, it is better to measure 1 x 1 meter or 1 x 2 meters then to check squareness for the remaining distance using angular optics. It is very possible and true that you can make a perfect circle by cutting a square, but the opposite is not true. Another method using an interferometer is to use straightness optics then the pentaprism, but usually it’s too much of a pain in the tail and time consuming.  Interferometers cost about $50,000. CMS North America owns 3 and the factory several more.

Method: Ball Bar

Lastly, we need to talk about the ball bar (Sigh!).  Ball bars are very accurate linear encoders. Their general use is to be placed between a machine spindle and a fixed point then moved in a circle. As the circle movement is performed the distance between the spindle and the fixed point is recorded. The length of the ball bar can be adjusted by adding lengths of rod between the spindle and the fixed point. Many of these lengths can also be calibrated to exact lengths others are just a relationship between the axis’ movements.  Ball bars are wonderful instruments and are incredibly versatile; but after talking to some of the manufacturers’ representatives I was as confused as the jurors after the closing arguments at the OJ trial.  After many in depth discussions with respected metrologists throughout North America and countless experiments, I have come to the conclusion the thing the ball bar does worst, is squaring a machine! The way a ball bar measures squareness is that it takes the readings at the 4 quadrants starting at 45 degrees.  So in theory it is measuring diagonals! The problem is there are many other things going on at the same time with dynamic measurements. There could be vibration, stick-slip, lateral play, servo mismatch, scale mismatch and a host of other things. The manufacturers claim their software can lift and separate these items, but it’s just not so.  The other drawback is the size of the circle. I would never even consider any results from this measurement unless it covered 80% of the axis travel.  The manufacturer doesn’t recommend anything larger than a 300 mm radius. At this point weight and deflection become too much for good readings. I have and will continue to use ball bars, just not for squareness issues.  Ball bars cost about $15,000.

Method: Articulated Arm

One other method worth mentioning is the use of articulated arms or portable CMM’s to directly measure the machine tool and not the parts.

Limitations: Circle or Square Methodologies

Back to the circle and square. Nearly all machines are square or rectangular.  I am able to put the largest circle the machine is capable of cutting into the square that comprises the machine, but I am not able to put the largest square the machine is capable of into that circle.  Hence, I can make a perfect circle by making a square, but not the other way around.


To make a final summary, the best way to measure machine geometry is with a laser tracker and a skilled technician.  The amount of data that can be collected, the ability to compare thousands of points into best fits, and to measure plus and minus extremes in all directions make the tracker the best all-around measurement tool for medium to large machine tools. I hope to have one very soon!