In this article I talk about the things that I've learned about the academia - what kind of a market it is, what's the user profile and what kind of opportunities there are. This is an overview summarising learnings from my own multi-year experience in academia as well as 3 years+ research while building Nodebook and subsequent products, including many interviews that I had with people close to academia, tracking social media interactions, researching competition and related industries, reading papers, blog posts and books on academia, and implementing things myself.
The state of the academia
Academia is big, disogranized, costly, outdated, badly incetivized, hard, overwhelming, slow, and inert. Work in academia is repetitive, ugrateful, and anything you want to achieve is either very difficult or impossible. Academia is not about good science.
Let's understand each of these:
Big, disorganized, overwhelming
Academic market is huge. Just the academic publishing is a $37 billion market, yearly, with estimated 7% year-over-year growth. There's ~35,000 journals, ~27,000 higher education institutions, ~12,000 non-profits. It's a community of 300 million people, out of which there are 22 million people with PhD. Close to 2.2 million new papers get published every year.
With that many constituents, each with their own motives and goals for survival, each pulling in their own direction, it gets very easy for an entire system to feel disorganized and without unified direction.
For individual researchers, besides just being exposed to that many options to choose from, by often needing to synchronize the work between the parties, it gets overwhelming very easily.
Bad incentivies, slow, inert
For groups and for individuals - two-phase
because there is now incentive for change and there's too many parties, the change is very slow
conclusions that we can derives is that academia is very disruptive market. it's big and outdated, and it's target demographics is suffering badly.
one bad thing is that it's also a monopoly
Research cycle - types of products
The average person in academia is very tired, frustrated and probably always on the edge thinking whether to leave the academia and go into the industry.
How do you even approach to offering new products for something like that
Recenlty, I've been gather more information about the state of academia from organizational point of view, and not from the point of view and needs of individuals. Short summary would be: academia is disorganized, overwhelming, repetitive, unsatisfying and ungrateful, and anything that you want to achieve is either very difficult or impossible.
There're thousands of paper repositories. There's hundreds (maybe thousands?) of organizations pushing their own agendas, work, writing and retweeting articles like crazy. There're tens of tousands universities with their own curriculums and opportunities, tenures, fellowships, internships, collaborations. Tens of thousands of journals with their own desires, rules, formats, deadlines and restrictions.
I wondered how is it possible that academia manages to make any progress at all under those circumstances?
The thing is, some progress more or less has to happen. When you type something into a search engine, some result will come up. That result most probably isn't the ideal solution to your needs, but it's "good enough". If it's not even good enough, then you spend more time searching, all the while being practically blocked to continue with your progress and at some later points in future you'll be able to continue.
There's a huge problem with filtering content in academia, so far unsolved, but more on that concretely on some other ocassion.
It's still very interesting how people close to academia try to solve problems that they experience. Another coallition.
It's all about incentives
Non-profits lack incentives to do really important things and to achieve them efficiently.
I always think of an example with dentists recommending a brand of toothpaste. Yes, dentists care about mouth hygene and want to see their patients
There's basically a conflict of interest. For non-profit to exist, a problem that it's trying to solve needs to exist. That is true for both for-profits and non-profits. So far so good. But the problem starts with
Two-phase method for global optimization
Optimization is generally a process (an alrogithm) of finding and picking the best candidate as a solution to some problem out of all the other alternative candidates. Set of candidates is called a "search space". Global optimization is when you consider entirety of search space and pick the very best solution. Local optimization is when you consider only a subset of the entire search space, and pick the best solution only from that subset (called "local" because candidate solutions from subset search space are usually heavily coorelated by some property, their position, or similar metric for their "distance"). Global optimization finds "global optima", while local optimization finds "local optima". It might happen that a local optima is at the same time a global optima as well, but that is unknown.
Two-phase method is one of many potential algorithms for global optimization.
And why is any of this relevant to our discussion? Because human endeavors are all optimization algorithms. People behind academia are basically doing an optimization of the entire scientific pipeline. Their goal is to consider all the possible needs and variations of how science is and could be done, and to propose a framework and pipeline under which all of that can happen most optimally. We aim for more optimized solutions because practically that means faster iterations and more progress, smaller costs, and faster time from theory to application.