Fat startup: Learn the lessons of my failed Lean Startup — Word Sting.by John Finneran, CFA principal of Word Sting
I wanted to comment on the post linked above, because 1) I am working a bit with the “lean startup” model with student project groups, and 2) I am always a bit skeptical of “flavor of the day” business jargon. I found it interesting that this was posted online shortly after the Harvard Business Review published “Why the Lean Start-Up Changes Everything” by Steve Blank. This release, in the May 2013 HBR, heralds the alleged “entry” of the lean startup concept into “mainstream” business concepts. While the two posts aren’t directly relevent, John Finneran’s brief description of his own experience in attempting to “implement” lean startup seems instructive at one level–even if “lean” is the right way to go, it might not be easy to do it “right.” It isn’t clear how much of the story might be due to incorrect implementation of the concepts, or whether the concepts are not broadly applicable to all situations. In my own brief experience, I hesitated to fully embrace the full lean startup methodology due to my own concerns along those very lines.
On the other hand, I chose to use the concepts in exactly the same context that Steve Blank implemented them–working with university students who are engaged with “first time” startup efforts. These students are engaged in learning activities, and not primarily in the startup activity (regardless of the rhetoric around the subject). I found the business model canvas and the very slick exciting MOOC of Lean Launchpad to be useful tools for students with little or no experience in business research. In doing so, I hoped to leverage the fame and popularity of “startup party” mentality, hoping this would be motivating to students. I also found this to be an efficient direction, since I don’t have a burning desire to set up my own “school” of thought on the subject. However, I am nothing if not practical, and for me, this model is just one way to have a standard methodology for asking the right questions.
In our student team projects (exploring real-world application of ideas to develop new products or new businesses) there are fewer of the downsides expressed in the “Fat Startup” article. The “potential clients” who are contacted understand that the ”startup” is really part of a class project. The students, no matter how serious they are about moving forward with the new venture, still expect a ‘grade’ for their final efforts. I many of the projects, the effort is being made too early for real “customer” interactions. These students are doing research, and there is no doubt about it. In fact, my expectation is that the majority of the projects will result in “failed” experiments. I try to prepare the students for this by telling them so–but I also tell them they will learn very valuable and practical lessons that they can use when they later find the right path to a startup (or development of a new product).
These projects offer a form of “real world” experience, but maybe not so “real” as that. The reality is in the tasks, but not in the context. The so-called enterprise may be little more than a group composed of college friends and roommates. They generally fit this activity into their overcrowded calendars in the same way they might do for fraternity or sorority functions (whether service or social). I can suggest that they use the Lean Startup methodology for their projects since I find it has some solid elements of logic, but I’m not concerned they will rely on this effort to support themselves (or any dependents). And they aren’t in a position to make promises to customers, just take suggestions and see if they can find a path that looks promising…but they really are planning a future journey. Their project might involve ”getting to the station” as their first “step” and they still have the option to take another train (or even consider an entirely different mode of travel, maybe a boat).
Don’t get me wrong–I wouldn’t suggest the model to my students if I didn’t find it valuable. I just want them to use their own judgement, and learn how to THINK before they make decisions. This includes the decision on whether, or not, to rely on something like the “lean startup” concept for taking that leap into the deep (and muddy) waters of entrepreneurship. University faculty are in much the same position as students, and they are being given direction to use the lean startup model in their own new ventures (for example, NSF Innovation Corps (I-Corps)). I feel this might be useful with some faculty, those who can truly put themselves back into “student” mode for the purposes of learning a subject (business or entrepreneurship) of which they can comfortable admit ignorance. It might not be applicable to all new ventures or to all faculty entrepreneurs.
The moral of the story? Just be careful about jumping on passing bandwagons.
Both of these short “editorial” pieces ask an interesting and somewhat philosophical question about the role of the university in student “entrepreneurship.” The linked articles (by Nicholas Thompson, editor at Newyorker. com and Stanford alum) question some of the more “intense” levels of involvement that many universities are embracing, with schools like Stanford as our role models. Now, I’ve always tried to resist using “what Stanford does” as a yardstick–I even included a nice little slide in one of my presentations, showing little lemmings jumping off a cliff with a cartoon balloon from one shouting as it fell “…well, Stanford does it!” But the urge to follow their lead is almost irresistible.
At its heart, this is nothing more than a bit of “peer pressure” since every university is looking to establish a reputation–presumably a “good one” too–showing how their own activity reflects the “best” new practices. In some cases, it may be near impossible to replicate these practices (we’re no Silicon Valley) so at least that provides some protection from following all the yellow brick roads “we” see. It’s easy though, to wish your own institutions of higher education could collect all the accolades that a school like Stanford receives.
After some reflection, I realized that the piece reminds me a bit of the book I’m reading” (audiobook format as usual these days, so “listening to” is more precise) “Antifragile” by Nassim Nicholas Taleb (click here for his website and here for the Amazon link to the book–note I am not an Amazon affiliate, just being helpful). Now, that is a bit of a leap, but I tend to think that way, so it’s not surprising if you know me very well at all!
In the book, which I confess to not having finished as yet, Nassim Taleb refers to a phenomenon he calls “lecturing birds on how to fly” and makes some very disparaging references to Havard Business school faculty (not Stanford faculty, but I’m betting he would do so if he wasn’t more centered on the east coast). In many ways, his argument is also along the lines of a sort of “chicken and egg” situation (even though chickens don’t fly). As I see it, a school like Stanford provides a lot of opportunities that “attract” very entrepreneurial students, very smart ones at that, and then the “university” manages to get those little entrepreneurial birds to fly. Many other universities then follow Stanford’s lead on how to teach their own students the Stanford way of entrepreneurship, since it has such great success.
If you look at it that way, it seems that some of the things critiqued may be simply circumstantial, and Stanford isn’t really “doing” anything except meeting some expectations of the students already there. Is this “harmful” to Stanford in some way, as Mr. Thompson is pointing out? Perhaps, but it may be more true to say that Stanford is being given credit for ‘causal’ influence that isn’t there. In some cases, maybe there is also a negative impact on students (or more precisely, on their education). Certainly, I think it’s a good idea to ask the questions…but is someone going to try to find an answer? Regardless, it serves as one more example for me to use when people try to propose the next “me too” model for my own programs or projects.
This is hard to resist–who among us doesn’t feel the need for validation that takes the form “well, I found out that X is doing things this way”? As happened regularly, I was once contacted by a colleague at another university (no names), who was engaged in a contract negotiation with a company. The company had cited my own university as an example of a peer who had “agreed to these terms” so she called me to confirm. One of the reasons she did so was that the terms seemed to be “less desirable” and she seemed to suspect the company was not telling the entire truth. Now, at this point, I didn’t want to get into exactly what my organization had (or had not) agreed to, since there were too many complicating issues to explain. But I did give her one piece of advice that seemed to help. I told her, “Just go back with–even if “University X” agreed to those terms, that’s not a good reason for us to agree.” As I explained, there might BE good reasons for her to accept those terms, but the company needed to work through those specific reasons with her–not just “by reference” insist that since another university agreed, it was OK.
Otherwise, I am continuing to “read” the Taleb book, and I hope that there are some more interesting issues raised. As I understand, Nassim Taleb is nothing if not interesting, and I’m looking forward to learning more about some of his insights on innovation.
This article argues that perhaps “we” (as in, the administrations of universities, and the government) are trying too hard to push research commercialization.The argument is primarily against special spending for programs and financial incentives to faculty encouraging them to pursue technology transfer. It is an interesting take on the question of university technology transfer…and I admit, I’m more than a little sympathetic to the views expressed. Which is unfortunate for me, since there are some who feel that my current role within the university is to find ways to “increase” commercialization levels on our campus.
Granted, I don’t consider this to be the essential role I play in this regard, but others would consider it a “failure” on my part if some metrics related to commercialization don’t “increase.” I tend to take a more balanced approach. I don’t want to “increase” commercialization, per se. I am working to reduce or remove roadblocks to commercialization when there is a good opportunity to take some research findings to market. There may be unnecessary difficulties–generally caused by either lack of experience on the part of faculty or administrative or financial issues that limit the perceived upside to pursuit of commercialization. The article does a good job outlining the failure of “additional” incentives, and points out that the natural incentives exist for those cases where there is a big payoff–and as I often point out to faculty, those tend to be commercialized in spite of so-called barriers to the process. Most faculty aren’t overly influenced by the money argument anyway, although it falls under the category of “great if it happens.”
I’m not sure the article does justice to another aspect of the problem however. There are instances where there is a clear benefit to taking the research outcomes and making them available in the “real world” but there is not a clear “profit motive” to do so. Faculty may be very interested in seeing the results of their work applied and in being part of the effort to make this happen. They are doing research ostensibly to help “solve” problems in many cases. Often, this leads to a lot of confusion as to how the university might proceed. There a quite a few options to consider, such as:
- Publish the results and make them “freely available.”
- Pursue some kind of “open innovation” strategy that includes a level of intellectual property protection (such as open source licenses for software).
- Try to pursue commercialization through entrepreneurship, which may not include licensing of intellectual property.
- Pretty much do nothing but sometimes think about it, etc.
All of the options while somewhat straightforward can lead to paralysis by analysis and other common forms of deadlock. Especially if the people involved (the research team) are inexperienced and unfamiliar with the process. The researchers have a goal that is consistent with technology transfer and licensing, but without all of the pieces coming together properly.
For example, their response may be to publish, but this winds up being unsatisfactory because there is no clear transfer of an idea into use. If there is a decision to hold off publishing, it may be that there is a naive belief that they will be able to enter into a commercialization relationship, and if they can’t it wouldn’t be fair if “someone else” got to make money off of the “idea” from the university. Conversely, potential partners may be confused as well, if there is not a clear IP position that can be evaluated so they can make a decision to “license or not license.” There may be some strategies that the faculty can pursue in these cases, but as the article in Business Week points out, adding incentives doesn’t solve them.
The article further references an earlier publication, by Richard Jensen of Notre Dame and Marie and Jerry Thursby of Georgia Tech titled “Disclosure and licensing of university inventions: ‘The best we can do with the s**t we get to work with.’” (click here for this paper). The Businessweek article notes the following (emphasis added):
The title, taken from a comment made by one of the licensing officers, sums up what happens when you give universities an incentive to commercialize additional faculty inventions but you can’t do anything to improve the quality of the inventions themselves.
Now, you can perhaps characterize those situations that I refer to above in this way–the “inventions” are not of proper “quality” to license. As a licensing manager, I can help faculty work through the problems and see if there really is a diamond in the rough, but we can’t make a diamond out of a piece of coal. But notice, a piece of coal can still be worth something. And it can still be important to someone, even if there is no clear-cut way to “commercialization” for that.
There are incentives for most licensing offices to work ONLY with faculty who have the “quality inventions” and no incentives for those same licensing officers to find a way to “work with” the so-call “s**t” as it were (note, I don’t approve of a licensing office using that attitude when referring to an invention disclosure–although I can feel a bit of empathy). So perhaps there is some room for universities to take some steps to encourage efforts at commercialization, and some of this may be in the form of incentives (although perhaps changes to incentives for licensing staff, not the faculty). The moral of the story is that incentives aren’t necessarily the best answer in all situations, and as they say…be careful what you ask for because you just might get it.
Again, I find myself inspired to post here, after reading one of Gerald Barnett’s most recent blog postings, Five Defects in Persistent Readings of Bayh-Dole | Research Enterprise. Early in the post he makes a statement that really struck a nerve with me (emphasis added):
I made this list of serious defects in characterizations of Bayh-Dole in the academic and popular press. Why do these persist? It must be that there are folks who really want to promote defective readings of the law.
Why did this strike me in particular? It took me a few minutes to think that through. First I realized that my experience in technology transfer was a key factor. In the first few years of my career, I certainly mirrored exactly the same “reading” of Bayh-Dole because most of the training and mentoring I received was from people who understood the legislation in this manner. It is part of “our” professional identity to understand and communicate the essence of the legislation that was the foundation upon which our technology transfer offices were built.
After those first years, I became less concerned with any “essential” truths on Bayh-Dole as my job focused primarily on details–making decisions on patent applications, reviewing patent prosecution correspondence, meeting with new inventors, negotiation of licenses (or executing the more common agreements, non-disclosures, MTA’s, etc.). Professional development centered upon attending conferences and workshops which either 1) repeated the conventional wisdom on Bayh-Dole, or 2) focused narrowly on the technical side (patent law, etc.). It isn’t unusual for anyone to get caught up in the day-to-day activity and feel little or no interest in reflection upon something that has ceased to be a question.
Unfortunately for me, I am often “accused” of over-thinking on pretty much any issue or subject. Also, I am more inclined to question conventional wisdom than most. I am not particularly impressed with arguments that have little in the way of DATA to support them. Of course, I wouldn’t t let that get in the way of the daily routine. I managed to continue with the meetings with inventors, whether (or not) I “believed” in all the myriad “assumptions” of the legal foundations and how the legislative underpinnings were supposed to work.
Admittedly, I was sometimes at a loss when asked to really “justify” some of the policies and processes I was expected to operate under. I could honestly agree with, or at least provisionally consider, many of the opposing viewpoints–for example, why did the university policy require faculty to assign all patent rights to the university? How can the policy state that faculty will own copyright in most of their work, but then add provisions where the university claims a specific work?
If I had reservations about whether these issues were being dealt with “rightly” I mostly kept them to myself. After all, if I raised doubts with the wrong person, their conclusion might be that I didn’t really understand my job, especially if they bought into the conventional thought on the subjec.t In a few instances, when I did at least highlight some of the more problematic points, there was little interest in making changes–and for a variety of reasons. For example, policy was too difficult to change, and no one understood it anyway. Besides, there I was again–thinking too much, and asking too many questions.
This gets to the heart of why I pretty much left well enough alone. It was easier to keep on track with everyone else, and I didn’t really have the energy (or the commitment) to do more. I focused on the technical side, glossed over those pesky “principles” and just tried to approach each projects from a common sense point of view. It could be fairly uncomfortable, but I’m generally more tolerant of that sort of ambivalence.
With respect to Bayh-Dole, I’m willing to bet that many others in same position will either 1) avoid questioning conventional dogma, or 2) insist upon the innate truth of said dogma. Perhaps this does result in their promotion of “defective readings of the law.” I may find it regrettable, but I also find that I can sympathize with them to point. It’s hard to come to an unconventional conclusion and attempt to stick to it–especially if there is some chance that you will come to an incorrect conclusion, or at least a certain percentage of people will judge you to be “wrong.” Even when I do find myself coming to unconventional conclusions, I often default to discreet silence and not publicly expressing my doubts. However, by my silence, it’s possible to conclude that I am contributing, albeit passively, to promoting the opinions that others are more aggressively voicing.
This was the point that made me pause in reading Gerry’s original blog entry. What, precisely, do I really “believe” and what should I then do about it? I have to think more about the subject, and that I will leave for another posting.
Infographic Created by: ClinicalPsychology.net
This posting isn’t particularly related to technology transfer, and not even strictly speaking to intellectual property. But I found this infographic to be quite interesting, and it struck a nerve with me, especially due to the recent news coverage of Jonah Lehrer‘s unfortunate decisions in writing his latest book (for reference and to read more about the story click here). There is a common theme to both that leaves me troubled and sympathetic at the same time.
The troubling aspect is that there are certain individuals who apparently think some ”rules” apply to other people, but who are more flexible when applying these to their own situation. This is true even for some our society would consider the most well educated and intelligent. Perhaps they feel their own intelligence is a license to make a “judgment call” about whether, or not, a rule is a good one. Maybe they even convince themselves that they are correct in the “spirit of the law” as it were, and that others don’t comprehend all of the nuances of their “particular” situation. They may claim to see no rational justification for a particular rule.
Nonetheless, the rules, or the laws, exist for a reason.
Even IF you can make an argument that a particular rule or law is inherently flawed, that doesn’t justify making a decision to break it unless you also acknowledge this is what you are doing. I have no problem with someone taking a stand against an unjust law or an unreasonable and arbitrary rule. But when the rules or principles of good scholarship are violated by a researcher as described in the infographic, it undermines the foundation on which other researchers are attempting to build new knowledge. Researchers who do this are intentionally making choices to enhance their own professional status, to add achievements or successes to their reputation that are fraudulant. In this situation, researchers are allowing self promotion to trump the social contract that is essential for the research community. The scientific research enterprise has evolved such that researchers are meant to benefit from the work of others, in order to advance the overall body of knowledge in a particular field. There may be situations in which, truly, one is only hurting oneself with bad decisions, but this is not one of them.
The sympathic response is just a reflex, as I realize that in so many cases these researchers–academics, scientists, scholars–are only human, and it is so easy to slip into these habits. Most of the training or mentorship that people receive is focused on field specific knowledge (Jonah Lehrer, for example, has an undergraduate degree from Columbia with a major in neuroscience). Students are expected to master large bodies of scientific or technical knowledge, while some of the ethical and/or philosophical principles of scholarship and research is given little emphasis.
Even when someone with a great deal of interest and motivation to learn and understand attempts to grapple with some of these issues, it can become confusing. What is the difference between copyright infringement and plagarism? Can you truly “plagarize” your own work? For me, it’s natural to assume that someone didn’t really “mean” to do something wrong, as I like to think the best of people, especially of those engaged in scientific or scholarly research. But there seems to be an element of defiance in some of this behavior. Given my experience in academia, I repeatedly encounter students who explicitly discount some of the “older” traditions of scholarship. Their notions of how to appropriately “paraphrase and cite” another’s work are often made on the assumption that it doesn’t matter that much. Frequently, they will even defend some of the most egregious examples of plagarism (perhaps, copying entire sections of a Wikipedia article and “changing the words”). Their position appears to be that they are merely playing an intricate game, and that the end results (the appearance of “expertise”) justify their means.
I’m sure that there are multiple levels of self justification and self deception that allow researchers to make this kind of decision. They may think (or rather “feel” since I can’t say they “think” much at all on this subject) that this is not so different from driving 75 mph in a 70 mph zone–as long as no one gets hurt, and you don’t get caught, what is the harm? If you go 85 mph, is it that much worse? Certainly there are all the earmarks of a slippery slope that will lead the unwary or lazy researcher into a spiral of unprofessional and unethical behavior. Likely, each little step seems more akin to a “white lie” and not anything ugly or immoral, like fraud.
Still, it is important for everyone–both as individuals and as a community–to see that the rules and laws that we put into place are good ones, and that everyone understands that it is important to follow them. The rules must truly support the common good, and thus if you follow them it is more beneficial than breaking them. Even if it seems that breaking the rule will be “better” (for your own benefit at least) this isn’t truly the case. This is one of the most basic lessons that all good parents attempt to teach their children–yes, the homework is hard and skipping it “feels” better now, but in the long run, you are better off doing it. Even if you think that it won’t hurt anyone, even if you probably won’t “get caught” it’s still important.
Jonah Lehrer was by all accounts, considered a gifted and insightful writer, but now, each of his prior achievements in life is suspect, so he is certainly paying a price for his own poor judgement. Perhaps, as a journalistic writer, his actions didn’t undermine a great deal of “serious” scholarship or research, but there is still a cost to society. How many people bought his book with sincere interest and respect for his reputation, only to now feel a sense of betrayal? They spent some small amount of their own resources on this, perhaps not a huge loss to most of them, but they were defrauded at least to some extent. For the scientific misconduct described in the infographic, the cost may be unmeasurable. A scientist who falsifies data may be guilty of steering others away from promising directions that could result in tremendous advances, or lead them afield through false leads, wasting valuable resources in vain pursuit of something that was never there. Thus, my tiny impulse to sympathy is short lived.
If someone is truly confused, and not simply feigning this as a rationalization for their choices, then the answer is to be more open about these issues. We should be more explicit in teaching students how to properly conduct themselves in the course of research and publication. This can include more education on how intellectual property rights figure into the equation, but it is important to realize that this is more than an issue of “property rights.” All of us could probably benefit from a bit more self awareness and reflection in their own professional conduct in this respect. I’m sure that many of these people who have admitted “misconduct” might, at one point or another, have written a very similar posting, and felt they would not fall prey to such temptation. Your own “bad habits” might seem small, insignificant but it’s important that you can say you actually made the effort to do the right thing, and not take advantage of the loopholes or rationalizations.
First thing, I’ve meant to be more consistent with posting, but got caught up in launching a small “idea pitch” competition for students. It was a good experience, and fun was had by all of course, but I didn’t see daylight for the past couple of months. However, as my schedule has returned to normal in the past week, I noted increased discussion in technology transfer circles on the subject of “free agency” and the rise of a more concerted “just say no” campaign. Ordinarily, I would steer clear of the subject—for me, it falls into the cateogry of a subject better left alone if there isn’t a good chance to have a calm and reasoned discussion. On the other hand, I’m not ever clear that the subject truly is “free agency” (whatever most people mean by that) or if it gets back to the “Stanford v Roche” questions on ownership of inventions made by university researchers. The two are, of course, linked. But ownership of intellectual property should be a question that can be answered factually.
I won’t make any attempt to revisit Bayh-Dole legislation, or the Stanford v Roche decision, and interpret those in any way–Gerald Barnett is doing this with much more rigor than I would be willing to devote to the effort. I would just say the answer is along the lines of “in a particular case, figure out who owns the intellectual property.” The owner of the intellectual property—whether by right of inventorship, by contract or voluntary assignment, or by any other means of acquiring ownership which you might imagine—is entitled to make decisions on how to manage the rights. Free agency is actually the default option when the inventor(s) own(s) intellectual property. But so what?
In the end, you simply have to ask, what are the real goals of university technology transfer? Intellectual property protection is really meant to help market investment, to develop innovations more quickly and efficiently. The party who makes the product/market development investment does so with some assurance that an enforceable patent will allow for at least a moderate period of uncontested market share—at least in theory. Since universities generally cannot take products directly into the market, and since an inventor rarely has the experience or resources to do so, licensing of patents is the default mechanism for inventions originating from university research. In my experience, the primary reason that faculty researchers are interested in technology transfer is to see that their knowledge is transformed into a real world solution to a problem. They choose to work with any licensing agent (whether internal or external) only if it seems there will be more benefit than bother associated with the activity–that is, when they feel strongly about getting a product developed and on the market. This may. or may not, include an expectation of significant financial rewards as well.
I’m very supportive of the licensing staff at most universities; after all, that is the role I’ve served for over sixteen years. I know all about the constraints and obstacles facing technology transfer efforts. I understand that a lot of the so-called “failure” there is rooted in circumstances that can’t easily be addressed (such as lack of funds to staff the office, or for prosecution of patents). Thus, it isn’t obvious that free agency will solve specific issues and problems with commercialization of university inventions, anymore than salads at fast food places solve the so-called epidemic of obesity. However, as much as we might hate to admit it, there are too many problems with management of intellectual property in a university setting to ignore. There may be lots of arguments against current proposals for so-called “free agency” but I find it difficult to oppose considering the model, at least in some form.
Even if free agency is an option, faculty would need to be motivated by other considerations to take advantage of their “freedom.” Since many of the free agency models are only dimly conceived—Who, exactly, are these “agents”? And how would they engage with faculty?–it would be necessary to think through the consequences of how a particular model was implemented. Many of the free agency proposals seem to assume that the “successful” university systems would be open to managing the patents for others. On that front, I cannot imagine that a university would allow their technology transfer office to work with another university’s intellectual property on a regular basis. Of course, joint ownership is an exception, but even then I’ve known it to work both ways. There are multiple cases where I was perfectly content to allow another university to “take the lead” but was met with resistance from the other side, where my counterpart was hoping my office would manage the intellectual property. It can be a great temptation to have someone else bear the expense, and be the bearer of bad tidings when a decision is made not to file a patent, etc.. Thus, it wouldn’t seem that free agency is always a bad deal for the technology transfer office.
So why not take a serious look at the model and see how it might be implemented to advantage in a particular situation? I doubt that this means a complete shutdown of the technology transfer offices. There are many facets of intellectual property management that the university will still want to deal with directly, and until there are more examples of specific routes for faculty to pursue, the existing licensing office will be the first stop–how else will faculty know where to go? It might not be a very attractive option for faculty when such a system is implemented at first, due to simple inconvenience and lack of other options. Of course, there are a few independent licensing groups, and sometimes attorneys can freelance this sort of work, but these aren’t simple options for the average faculty member to find and evaluate. Even if you work in this arena for many years, it can be difficult to engage with a suitable partner. Further, the terms of such a relationship might be a significant barrier—exactly how much does this option cost faculty?
Honestly though, I think it is worth the experiment. If the benefits of this kind of arrangement do outweigh the “free” option of the local university licensing office, how can you argue with this? I won’t, however, hazard a guess at this stage on how effective the free-agency model will be in practice. After all, if the primary and most obvious “agent” available is still the university’s own technology transfer office, or at least the closest “bigger” university office, it’s not clear this will result in great change. Still, I think there is room for innovation in technology transfer itself, not just in the research results from the university laboratories.