Force Design Blog

    Don’t Buy a Lemon! Tips for Smart Runoff Testing of Automation Equipment

    Nov 12, 2019 11:53:52 AM / contributions by Mark Miller

    Don’t Buy a Lemon! Tips for Smart Runoff Testing of Automation Equipment

    As a consumer you probably wouldn’t buy a car without test driving it. And you probably wouldn’t buy a new pair of shoes without first trying them on and walking or jogging in them at the store. You need proof a product meets your needs before you make a purchase, and the best way to do that is to test it under real-world conditions. It’s how you’ll discover a sluggish transmission or inadequate arch support or another deal breaker before you buy.

    This applies to purchasing automation equipment for your manufacturing facility too, and it’s why runoff testing is critical. Runoff testing comes at a point in the project when everyone is ready to wrap things up and start reaping the benefits of the new equipment. But it’s worth the time to plan and prepare for these final tests of the equipment so you can avoid faulty parts, drops in output, or downtime later. Let’s look at some tips for sound and efficient runoff testing.

    The goal of runoff testing is to make sure the equipment or machine does what was agreed to in the integrator’s proposal. This includes:

    • Making sure parts/assemblies meet specs
    • Ensuring equipment can run as intended for the needed amount of time without issue
    • Meeting any requirements for safety of operators and working with materials
    • Verifying that computer software interfaces, controllers, and settings make sense to operators and that they work correctly

    Your integrator should conduct two separate runoff tests: first, an extensive set of tests at their facility and then a final check at yours after installation.

    Planning Ahead

    It’s tempting to put off thinking about runoff testing, after all, it doesn’t happen until late in the project. When you stop to think that success during testing is the direct result of well-designed equipment, it makes sense to plan for runoff in proposal and design phases.

    The more information your integrator has about your needs, the better the odds of building a machine that meets them, and the fewer adjustments to make later. Explain your goals and requirements clearly and in detail. In addition to what the equipment needs to accomplish (e.g. weld two components, move parts from one place to another), be sure it’s clear why processes are structured as they are, how operators are involved, specs for the finished pieces, and any safety concerns.

    A comprehensive picture of the end product points to the most valid ways to test it. It also helps in establishing a plan for how testing will occur when the times come. A solid runoff plan covers these things:

    • Create a document listing acceptance criteria, how parts will be evaluated, and any special instructions. Record runoff data in detail and note any failures or changes.
    • Use actual parts, not a computer simulation, whenever possible.
    • Provide regular, quality-inspected parts to imitate real-world conditions.
    • Plan out how you’ll have enough parts ready in time for runoff (including production time and any extra expenses for material or staffing). Consider having more parts than the quantity specified in the proposal, in case on-the-spot adjustments or extra testing are needed.
    • Have all needed supplies on hand, such as compressed air, packaging, conveyors, carts, etc.
    • Plan your schedule and budget to have staff readily available during runoff (e.g. engineers, quality, safety, operations as well as machine operators).
    • Figure out how to get your staff to the integrator’s facility for initial runoff testing. Make any travel arrangements well ahead of schedule.
    • List any equipment needed for evaluating finished parts, such as measurement tools, gauges, or CMM equipment, destructive weld testing equipment, etc.

    At Test Time

    Runoff at Your Integrator’s Facility

    Testing should mimic real-life operating conditions to obtain valid results. Objective acceptance metrics are the best way to know if the machine is working as intended. Common metrics include the number of errors per shift, cycle time, the period of time the machine can maintain part quality, changeover time, the ability to run for a full shift, and process capability index (CPK), which measure how many parts fall within the upper and lower limits of specs. There may be additional metrics to include based on your specific project.

    Now’s also the time to test human-focused factors too. These are things like the amount of time or effort required to do a task with the machine, ease of use (of the equipment and any software), and safety concerns. Sometimes problems don’t surface until test time, and they impact your ability to use the equipment. For example, a sleek, efficient machine with high throughput is not a reasonable solution if it threatens to pinch a worker’s arm each time it’s loaded.

    If problems are identified, the integrator’s job is to make adjustments and test again. Your job is to make sure you’ve conveyed exactly what’s wrong or asked questions about any adjustments suggested. Just remember this is not the same as asking for new capabilities or major changes, which will likely be out of scope of the contract.

    Some types of changes outside the original scope can be accommodated if you’re willing to incur additional costs and delay final start up. Talk with your vendor to see what’s possible. In general, changes are easier to make while the equipment is still at the integrator’s facility.

    Once needed adjustments are made runoff is continued to ensure problems are corrected. If adjustments are minor, you may choose to forgo additional testing and simply have the vendor confirm all changes were made before shipping.

    Final Runoff at Your Facility

    By the time equipment is installed in your facility, bugs, safety issues, and operating problems should be resolved. At the time of installation, another runoff is performed. This is generally not as detailed or lengthy as the initial testing at the integrator’s facility – in most cases it’s done to check any adjustments made to correct for shifting during transport or to verify the steps robots are “retaught” once in their final locations.

    Again, you’ll need to have all peripheral equipment, supplies, staff, and operators present and ready to perform tests. This is another time when having a plan in place ahead of schedule will keep things moving along so you’re ready to get back to production right away.

    Runoff testing is just a portion of the customized service that an experienced automation vendor provides. Curious about the rest of the process? Contact us today.

    Your Guide to a Smooth Automation Project - Download the eBook

    Topics: Automation Projects

    Mark Miller

    Contributions by Mark Miller

    With over 20 years of experience in the industry, Mark is passionate about promoting excellence in engineering. He is able to accomplish this by managing Force Design's internship program and through his role as our project optimizer.