Limitations of Two-Dimensional Indicators and unconstrained radial DEA models in Evaluating Judicial Efficiency: Insights from Poland's Appellate Court System
DOI:
https://doi.org/10.12775/EiP.2025.20Keywords
judicial efficiency, appellate court system, (un)constrained radial DEA models , disposition timeAbstract
Motivation: Two-dimensional efficiency indicators provide only a narrow perspective on a system’s functional efficiency, a limitation that is addressed by multidimensional methods such as the widely used Data Envelopment Analysis. However, the uncritical application of this class of methods—which appears to be a common practice in judicial efficiency research—can result in misleading conclusions regarding efficiencies of Decision-Making Units (DMUs).
Aim: The aim of this study is to evaluate the technical efficiency of Poland's appellate court system in the first two decades of the 21st century (years 2002-2021) using a constrained version of Data Envelopment Analysis (DEA) applied to time-series data. Unlike two-dimensional judicial performance indicators, such an approach facilitates a comprehensive assessment of the aggregated efficiency of the legal, administrative, procedural, organizational, and economic macro-frameworks—consistently implemented across all appellate courts—which collectively shape the system’s performance during each year under review.
Results: The outcomes of this research show the fundamental shortcomings of classical applications of DEA, raising valid concerns about the reliability and interpretability of the outcomes derived from this approach. To mitigate these challenges, it presents a novel procedure specifically aimed at addressing these limitations in evaluating judicial efficiency, which is next successfully implemented.
References
Agrell, P.J., Mattson, P., Mansson J. (2019). Impacts on efficiency of merging the Swe-dish district courts. Annals of Operations Research, 288, 653-679. https://doi.org/10.1007/s10479-019-03304-0.
Allen, R., Athanassopoulos, A., Dyson, R.G., Thanassoulis, E. (1997). Weight restrictions and value judgements in Data Envelopment Analysis: Evolution, development and future directions. Annals of Operations Research, 73, 13-34. https://doi.org/10.1023/A:1018968909638
Andersen, P., Petersen, N. Ch. (1993). A Procedure for Ranking Efficient Units in Data Envelopment Analysis. Management Science, vol. 39(10), 1261-1264. http://dx.doi.org/10.1287/mnsc.39.10.1261
Avikran, N.K. (2015). Productivity in the Service Sector with Data Envelopment Analy-sis. SSRN Working Papers. DOI:10.2139/SSRN.2627576
Banker, R., Emrouznejad, A., Lopes, A.L.M., Rodrigues de Almeida, M., eds. (2014). Data Envelopment Analysis in the public sector. Socio-Economic Planning Sciences, 48(1), 1-104. https://doi.org/10.1016/j.seps.2013.12.005.
Boardman, A. E., Greenberg, D. H., Vining, A. R., & Weimer, D. L. (2018).
Cost–Benefit Analysis: Concepts and Practice (5th ed.). Cambridge University Press.
Buonanno, P., Galizzi, M. (2014). Advocatus, et non Latro? Testing the excess of litiga-tion in the Italian courts of justice. Review of Law and Economics, vol. 10(3), pp. 285-322. https://doi.org/10.1515/rle-2014-0022.
Revised Saturn Guidelines for Judicial Time Management (2018). CEPEJ, European Commission for the Efficiency of Justice. https://rm.coe.int/cepej-2018-20-e-cepej-saturn-guidelines-time-management-3rd-revision/16808ff3ee
Cook, W.D., Tone, K., Zhu, J. (2014). Data envelopment analysis: Prior to choosing a model. Omega, 44, 1-4. https://doi.org/10.1016/j.omega.2013.09.004.
Cordero-Ferrera, J.M., Pedraja-Chapparo, F., Santin-Gonzalez, D., (2010). Enhancing the inclusion of non-discretionary inputs in DEA. Journal of the Operational Re-search Society, 61(4), 578-584. https://doi.org/10.1057/jors.2008.189.
Data Envelopment Analysis in Public Sector (2014). Socio-Economic Planning Sciences, special issue, 48 (1-2)
Deynelli, F. (2012). Analysis of relationship between efficiency of justice services and salaries of judges with two-stage DEA method. European Journal of Law and Eco-nomics, 34, 477-493. https://doi.org/10.1007/s10657-011-9258-3.
Damodaran, A. (2012). Investment Valuation: Tools and Techniques for Determining the Value of Any Asset (3rd ed.). Wiley Finance
Dyson, R.G., Allen, R., Camanho, A.S., Podinovski, V.V., Sarrico, C.S., (2001). Pitfalls and Protocols in DEA. European Journal of Operational Research, 132, 245-259. https://doi.org/10.1016/S0377-2217(00)00149-1.
El Bialy, N., Garcia-Rubio, M.A. (2011). Assessing judicial efficiency of Egyptian first instance courts: A DEA analysis. MAGKS Joint Discussion Paper Series in Econom-ics. https://www.econstor.eu/bitstream/10419/56541/1/657907855.pdf
Falavigna, G., Ippoliti, R., Manello, A., Ramello, G. (2015). Judicial productivity, delay and efficiency: A direction distance function (DFF) approach. European Journal of Operational Research, 240, 592-601. https://doi.org/10.1016/j.ejor.2014.07.014.
Garcia-Rubio, M.A., Rosales-Lopez, V. (2010). Justice & Economy: Evaluating judicial efficiency in Andalusia. SSRN Working Papers. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=1738227
Gupta, M., Bolia, N.B. (2020). Efficiency measurement of Indian high courts using DEA: A policy perspective. Journal of Policy Modeling, 42, 1372-1393. https://doi.org/10.1016/j.jpolmod.2020.06.002.
Guzowska, M., & Strąk, T. (2010). An examination of the efficiency of Polish public sector entities based on public prosecutor offices. Operations Research and Deci-sions, 20(2), 41–57. IDEAS/RePEc
Ippoliti, R., Tria, G., (2020). Efficiency of judicial systems: model definitions and out-put estimation. Journal of Applied Economics, 23(1), 385-408. https://doi.org/10.1080/15140326.2020.1776977.
Kapelko, M. (2024). Evaluating input- and output-specific inefficiency in courts of justice: An empirical study of Polish district courts. International Transactions in Operational Research, 32(5), 2767–2797. https://doi.org/10.1111/itor.13503
Krajewski, E., Milewski, R., (2018). Podstawy ekonomii [Fundamentals of economics], PWN
Kittelsen, S.A.C., Forsund, F.R. (1992). Efficiency analysis of Norwegian district courts. Journal of Productivity Analysis, 3(3), 277-306. https://doi.org/10.1007/BF00158357.
Lewin, A.Y., Morey, R.C., Cook, T.J. (1982). Evaluating the administrative efficiency of courts. Omega, 10(4), 401-411. https://doi.org/10.1016/0305-0483(82)90019-6.
Maital, S., Vaninsky, A. (1999). Data envelopment analysis with a single DMU: A graphic projected-gradient approach. European Journal of Operational Research, 115(3), 518-528. https://doi.org/10.1016/S0377-2217(98)00178-7.
Major, W. (2015). Data Envelopment Analysis as an instrument for measuring the efficiency of courts. Operations Research and Decisions, 25(4), 19–34. https://doi.org/10.5277/ord150402
Malcarne, A., Ramello, G.B. (2015). Judicial Independence, Judges’ Incentives and Effi-ciency. Review of Law and Economics, 11(2) 149-169. https://doi.org/10.1515/rle-2015-0024.
Mattsson, P., Tidana, C. (2019). Potential efficiency effects of merging the Swedish district courts. Socio-Economic Planning Sciences, 67, 58-68. https://doi.org/10.1016/j.seps.2018.09.002.
Muniz, M., Paradi, J., Ruggiero, J., Yang Zijiang (2006). Evaluating Alternative DEA models used to control for non-discretionary inputs. Computers and Operations Re-search, 33, 1173-1183. https://doi.org/10.1016/j.cor.2004.09.007.
Nissi, E., Giacalone, M., Cusatelli, C. (2019). The Efficiency of Italian Judicial System: A Two Stage Data Envelopment Analysis Approach. Social Indicators Research, 146, 395-407. https://doi.org/10.1007/s11205-018-1892-5.
Ostaszewski, P. (ed.) (2020). Efektywność sądownictwa powszechnego – oceny i ana-lizy [Efficiency of the common courts – assessments and analyses], Wydawnictwo Instytutu Wymiaru Sprawiedliwości, Warszawa
Ostrom, B.J., Hanson, R.A. (2003). Efficiency, timelines and quality: A new perspective from nine state criminal trial courts. National Institute of Justice and the State Jus-tice Institute. https://www.ojp.gov/pdffiles1/nij/178403-1.pdf
Panwar, A., Olfati, M., Pant, M., Snasel, V. (2022). A Review on the 40 Years of Exist-ence of Data Envelopment Analysis Models: Historic Development and Current Trends. Archives of Computational Methods in Engineering, 29, 5397-5426. https://doi.org/10.1007/s11831-022-09770-3.
Pedraja, F., Salinas, J. (1996). An assessment of the efficiency of Spanish courts using DEA. Applied Economics, 28, 1391-1403. https://doi.org/10.1080/000368496327651.
Peyrache, A., Zago, A. (2016). Large courts, small justice! The inefficiency of the opti-mal structure of the Italian justice sector. Omega, 64, 42-56. https://doi.org/10.1016/j.omega.2015.11.002.
Podinovski, V.V. (2016). Optimal weights in DEA models with weight restrictions. European Journal of Operational Research, 254, 916-924. https://doi.org/10.1016/j.ejor.2016.04.035.
Ruggiero, J. (1997). Non-discretionary inputs in data envelopment analysis. European Journal of Operational Research, 16, 461-469. https://doi.org/10.1016/S0377-2217(97)00306-8.
Santos, S.P., Amado, C.A.F. (2014). On the need for reform of the Portuguese Judicial System – Does Data Envelopment Analysis assessment support it? Omega – The In-ternational Journal of Management Science, 47, 1-16. https://doi.org/10.1016/j.omega.2014.02.007.
Schneider, M.R. (2005). Judicial career incentives and court performance: An empiri-cal study of the German labour courts of appeal. European Journal of Law and Eco-nomics, 20(2), 127-144. https://doi.org/10.1007/s10657-005-1733-2.
Siemaszko, A., Ostaszewski, P. (2013). Efektywność kosztowa sądownictwa po-wszechnego [Cost efficiency of the common courts]. Instytut Wymiary Sprawie-dliwości, 1-57, https://iws.gov.pl/wp-content/uploads/ 2018/08/Efektywno% C5%9B%C4%87-s%C4%85downictwa-powszechnego-oprac-12-1.pdf
Study on the functioning of judicial systems in the EU Member States (2022). Euro-pean Commission for Efficiency of Justice (CEPEJ). https://commission.europa.eu/system/files/2022-05/part_1_-_eu_scoreboard_-_indicators_-_deliverable_0.pdf,
Świtłyk, M., Sompolska-Rzechuła, A., & Oesterreich, M. (2025). Efektywność tech-niczna sądów okręgowych [Technical efficiency of regional courts]. Prace Naukowe Uniwersytetu Ekonomicznego we Wrocławiu, 69(3), 133–145. https://doi.org/10.15611/pn.2025.3.10
Tulkens, H. (1993). On FDH efficiency analysis: Some methodological issues and appli-cation to retail banking, courts and urban transit. Journal of Productivity Analysis, 4(1/2), 183-210. https://doi.org/10.1007/978-0-387-25534-7_17.
Voigt, S. (2016). Determinants of judicial efficiency: a survey. European Journal of Law and Economics, 42, 183-208. https://doi.org/10.1007/s10657-016-9531-6.
Voigt, S., El Bialy, N. (2016). Identifying the determinants of judicial performance: Taxpayers’ money well spent? European Journal of Law and Economics, 41, 283-319. https://doi.org/10.1007/s10657-014-9474-8.
Yeung, L., Azevedo, P. (2011). Measuring efficiency of Brazilian courts with data envel-opment analysis (DEA). IMA Journal of Management Mathematics, 22(4), 343-356. https://doi.org/10.1093/imaman/dpr002.
Yeung, L. (2018). Measuring Efficiency of Brazilian Courts: One Decade Later. SSRN Working Papers. https://ssrn.com/abstract=3200588
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Waldemar Florczak
This work is licensed under a Creative Commons Attribution-NoDerivatives 4.0 International License.
Stats
Number of views and downloads: 14
Number of citations: 0
