I am assisting in arranging a small conference in a highly mixed topic. There will be hard scientists, soft scientists, makers/creators/artists, researchers from many different fields etc.
I'm assisting in a number of small ways - one is how to decide on who should be able to present, and under what conditions/criteria. This is more difficult than I had imagined.
We will have no issue getting enough applicants, but we do have issues sorting through them. In the past, they've relied on a very quant-oriented process, however they've felt overall that this hasn't translated into the best pairing of speakers on panels, strength of tracks, etc. This year the proposal is to rely on more intuition, and qual-style evaluation (does it align with our goals, does it incorporate diversity, etc.).
Because we are quite small, there isn't the budget or capacity to get a wide panel of experts who can decide on quant-based merits for every applicant. We do have about 15 people who are active in the various tracks of the conference.
I'm trying to make this general enough to be helpful to others in the future, so the question remains simple: how does a very small conference evaluate conference proposals? Are there any best practices?