BUSI 600 Week 5 Group DB Forum 3: Part 1 Example

Group Discussion Board Forum 3: Part 1

BUSI 600 Business Research Methods, Group Discussion Board Forum 3: Part 1 (two threads)

Thread by [Student Name]

The two forum questions and the company examples are composites standing in for the course's own prompts; the research and Scripture cited are real, and the requirements follow Liberty's public course guide for BUSI 600.

Thread 1: Why a Business Measure Must Be Both Reliable and Valid

A business measure must be reliable and valid because consistency without accuracy produces confident mistakes. Reliability tells a manager that a measure gives the same answer under the same conditions; validity tells the manager that the answer is about the thing the company cares about. The measurement chapters in Schindler (2019) treat reliability as necessary but not sufficient for validity, and the order matters in practice.

Consider a composite hotel chain that built a guest loyalty index from five survey items: overall satisfaction, likelihood to recommend, cleanliness, staff courtesy and value for money. The items correlated highly with one another, internal consistency was strong and scores were stable from quarter to quarter. By every reliability standard the index was sound. Yet when the chain compared index scores with actual rebooking over the following year, the relationship was weak. Guests who scored the hotel highly were not much more likely to return than guests who scored it moderately. The index was measuring satisfaction with the last stay, not loyalty, and many satisfied guests simply chose a different hotel the next time because of location or a corporate travel contract.

Hinkin (1998) argues that most measurement problems in organizational surveys begin before any data are collected, with items written without a clear definition of the construct. His tutorial describes a sequence: define the construct, generate items from that definition, have judges assess whether each item fits the definition, administer the items, examine their structure through factor analysis, check internal consistency and then test whether the scale relates to other variables as theory predicts. The hotel chain skipped the first and last steps. It never defined loyalty, so it wrote satisfaction items, and it never tested whether the index predicted behavior until years later.

MacKenzie et al. (2011) make the same point at greater length in their ten-step construct validation procedure, insisting that a construct be defined conceptually before indicators are chosen and that researchers decide whether the indicators reflect the construct or form it. Loyalty is a useful example. If loyalty is a disposition that causes a guest to return, to recommend and to forgive a bad stay, the items should all reflect that disposition. If the chain instead wants an index of loyalty behaviors, such as the number of stays, program enrollment and direct bookings, those indicators form the index and need not correlate with one another at all. Choosing the wrong model leads to the wrong tests.

The fix for the hotel chain is to define loyalty, rewrite the items around that definition and then validate the new index against rebooking data. A reliable measure of the wrong concept is worse than no measure, because it carries the authority of a number.

Scripture repeatedly condemns false measures in trade. In Deuteronomy 25:13-15, Israel is forbidden to keep two different weights in a bag, one heavy and one light, and is promised long life in the land for keeping honest ones. A business that reports a satisfaction index as if it measured loyalty is not lying deliberately, but it is still using a measure that does not match the claim made for it. Validation is how researchers keep their measures honest.

Thread 2: Is a Convenience Sample Ever Good Enough for a Business Decision?

A convenience sample is good enough when the decision depends on discovering possibilities or on learning about the specific people sampled, and not good enough when the decision depends on estimating how common something is in a larger population. Schindler (2019) distinguishes probability samples from nonprobability samples such as convenience, judgment and quota samples. In a probability design, the chance that any given person in the frame will be drawn is known in advance and greater than zero. Only probability samples support statistical estimates of population values with a known margin of error.

Many business decisions, however, do not require such an estimate. A product team testing whether a new checkout screen confuses people can learn a great deal from twenty customers who happen to be in a store. If five of them cannot find the payment button, the screen has a problem, and it does not matter whether those twenty represent all customers. The decision is whether to fix the screen, and the convenience sample answers it quickly and cheaply. A larger, representative sample would add precision the decision does not need. Landers and Behrend (2015) argue that researchers too often dismiss convenience samples, such as online panels, on the basis of where they came from rather than on whether the sample's characteristics threaten the particular conclusion. Organizational samples, student samples and online samples are all convenience samples in some sense; the real question is whether the variables that differ between the sample and the population plausibly affect the relationship being studied.

When the decision depends on a population estimate, the standard changes. A regional bank deciding whether enough small-business owners want a new lending product to justify launching it needs to know roughly what share of its market would apply. Surveying owners who visit its branches would overrepresent existing customers, who may already be satisfied with the bank's products, and would miss owners who bank elsewhere. Here a probability sample drawn from a complete frame, such as a business registry for the bank's counties, is worth its cost.

Even probability samples can fail through nonresponse, and the evidence is more subtle than many managers assume. Groves (2006), reviewing 30 studies that measured nonresponse bias directly, found that the response rate was a weak predictor of bias. What produces bias is a link between a person's propensity to answer and the very quantity the survey is trying to estimate. A survey about interest in a lending product that mentions the product in its invitation may draw responses mainly from owners already interested, producing a biased estimate even with a high response rate. The lesson is to design invitations that do not reveal the survey's subject in a way that motivates some people more than others, and to compare respondents with nonrespondents on known characteristics.

The Law required that a matter be established by two or three witnesses rather than one (Deuteronomy 19:15). A convenience sample is a single witness, often one with an interest in the outcome. For exploratory work, one witness can open an inquiry. For a decision that commits significant resources on the strength of a population estimate, a manager should seek the fuller testimony that a well-designed probability sample provides.

References

Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 646-675. https://doi.org/10.1093/poq/nfl033

Hinkin, T. R. (1998). A brief tutorial on the development of measures for use in survey questionnaires. Organizational Research Methods, 1(1), 104-121. https://doi.org/10.1177/109442819800100106

Landers, R. N., & Behrend, T. S. (2015). An inconvenient truth: Arbitrary distinctions between organizational, Mechanical Turk, and other convenience samples. Industrial and Organizational Psychology, 8(2), 142-164. https://doi.org/10.1017/iop.2015.13

MacKenzie, S. B., Podsakoff, P. M., & Podsakoff, N. P. (2011). Construct measurement and validation procedures in MIS and behavioral research: Integrating new and existing techniques. MIS Quarterly, 35(2), 293-334. https://doi.org/10.2307/23044045

Schindler, P. S. (2019). Business research methods (13th ed.). McGraw-Hill Education.