All reviews of published articles are made public. This includes manuscript files, peer review comments, author rebuttals and revised materials. Note: This was optional for articles submitted before 13 February 2023.
Peer reviewers are encouraged (but not required) to provide their names to the authors when submitting their peer review. If they agree to provide their name, then their personal profile page will reflect a public acknowledgment that they performed a review (even if the article is rejected). If the article is accepted, then reviewers who provided their name will be associated with the article itself.
Dear authors,
Thank you for the revision and for clearly addressing all the reviewers' comments. I confirm that the paper is improved. Your paper is now acceptable for publication in light of this revision.
Best wishes,
[# PeerJ Staff Note - this decision was reviewed and approved by Stefan Wagner, a 'PeerJ Computer Science' Section Editor covering this Section #]
The manuscript is well-written.
The manuscript provides a comprehensive review of RL-HH, emphasizing technical and methodological detail that met to academic standards.
The discussion added.
The current revision is well-written and well-organized. I have no further questions.
Although it is not perfect I believe that this study can be published with its current form.
I have no further suggestions regarding the validity of findings.
Congrats
Dear authors,
Thank you for submitting your literature review article. Reviewers have now commented on your study. Your article has not been recommended for publication in its current form. However, we encourage you to address the concerns and criticisms of the reviewers and to resubmit your article once you have updated it accordingly. Furthermore, adding a comprehensive discussion for synthesis of findings, implications, future research, and limitations will be better.
Reviewer 2 has asked you to provide specific references. You are welcome to add them if you think they are relevant. However, you are not obliged to include these citations, and if you do not, it will not affect my decision.
Best wishes,
**PeerJ Staff Note:** It is PeerJ policy that additional references suggested during the peer-review process should only be included if the authors agree that they are relevant and useful.
**PeerJ Staff Note:** Please ensure that all review, editorial, and staff comments are addressed in a response letter and that any edits or clarifications mentioned in the letter are also inserted into the revised manuscript where appropriate.
**Language Note:** PeerJ staff have identified that the English language needs to be improved. When you prepare your next revision, please either (i) have a colleague who is proficient in English and familiar with the subject matter review your manuscript, or (ii) contact a professional editing service to review your manuscript. PeerJ can provide language editing services - you can contact us at [email protected] for pricing (be sure to provide your manuscript number and title). – PeerJ Staff
The manuscript is written in clear and professional English. The manuscript offers a comprehensive introduction to the field of RL-HH. The authors committed to the journal's format and scientific research structure. The manuscript falls within the scope of the journal. The authors acknowledged the existence of HH studies but utilizing RL in HH offers a recent and demanding perspective in the rapidly evolving area. The authors succeeded in providing an effective introduction based on the motivations for and advancements in RL-HH. Given the nature of the manuscript, it focuses on summarizing and categorizing existing research.
The manuscript provides a comprehensive review of RL-HH, emphasizing technical and methodological detail that met to academic standards. The manuscript shows a systematic literature search method for thorough and unbiased coverage of RL-HH, ensuring the possibility of replication. The manuscript from introduction through to conclusion, alongside appropriate source citation and use of illustrative figures and tables, contributes to a reasonable presented overview of RL-HH current state and advancements.
The manuscript addresses the impact and novelty of RL-HH by detailing the integration of reinforcement learning with hyper-heuristics, suggesting it fills a significant knowledge gap. Conclusions related directly to the research question, summarizing state and advancements in RL-HH concisely. It builds a strong argument based on systematic categorization and analysis of RL-HH, aligned with initial goals for a comprehensive understanding of the field.
Incorporating a comprehensive discussion section into the manuscript on RL-HH could substantially enrich the work. This section would serve to merge insights from value-based and policy-based RL-HH studies, emphasizing key themes and differences, and critically analyzing the implications for theory and practice. It would also offer a more nuanced look at future research course and critically assess the field's current limitations.
The paper is well-written and well-organized.
I believe that this research can be extended by including real-life applications.
Therefore, an additinal section that is devoted to real-life applications can be opened.
There is not a clear distinction between HHs and RL. In many papers these are used as interchangable terms. Maybe the authors should mention about this.
Before diving into details, some fundamentals of RL algorithms should be provided. For example, Q learning and Bellman eq should be given at earlier sections. Moreover, the differencens between SARSA and Q learning should be clarified. Pseudo codes should be provided.
I beleive that there are some missing studies that should take place.
**PeerJ Staff Note:** It is PeerJ policy that additional references suggested during the peer-review process should only be included if the authors agree that they are relevant and useful.
-Q-learning and hyper-heuristic based algorithm recommendation for changing environments
-Artificial search agents with cognitive intelligence for binary optimization problems
-A reinforcement learning based computational intelligence approach for binary optimization problems: The case of the set-union knapsack problem
-Iterated greedy algorithms enhanced by hyper-heuristic based learning for hybrid flexible flowshop scheduling problem with sequence dependent setup times: a case study...
-Evolutionary and population-based methods versus constructive search strategies in dynamic combinatorial optimization
All text and materials provided via this peer-review history page are made available under a Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.