I don't know of any module configuration that would do what you need without creating product variations for the samples. Now, just creating a variation and letting the custom select that on the Add to Cart form would be one way to make this work. I'm not sure what constraints you have that make this untenable, but you wouldn't even need to use product attributes if you didn't want to - just title the variations "[Unique Fabric] Roll" and "[Unique Fabric] Sample" and the Add to Cart form will present a select list for the variation to be selected by title.
That said, to do this via a generic product, I'd create a product + variation type called "Fabric Sample" that uses a new order item type to add the product to the cart. This order item type would have a reference field on it that I'd populate from the custom purchase link / form. You then would need to populate that reference field value based on the product the customer is requesting a sample for. The reason Commerce Add to Cart Link won't work is that the module doesn't have any way for you to provide values for custom fields on the order items it creates. That would be a fine feature request for the module, though. Otherwise, you're just going to need a custom form.